Merge lp:~vila/selenium-simple-test/test-loader into lp:selenium-simple-test

[Vincent Ladeuil]

Implement a test loader and options to include or exclude tests to run

This implements test filtering by adding two new options to sst-run: --includes and --excludes.

They allow one to run a subset of the tests by specifying one or several prefixes matching the test ids. Test being organized as a tree, using prefixes is simple and powerful way to reduce the scope of the test run.

This required a new test loader that address some limitations in the stock unittest one required for sst specific needs.

Those limitations are mainly two:

- scripts cannot be imported at load time (sst specifically compile them
  instead),

- unittest.discover defines a single pattern for matching both directories
  and files (sst has different rules for each).

Moreover, even providing separate patterns for dirs and files is not enough. sst allows scripts and regular tests to be mixed in the same tree.

The proposed loader offers a way to define specific rules for a subtree by providing them in the __init__.py file.

The limitation is that the same directory cannot contain both scripts and regular files unless the __init__.py file defines which are which and load them appropriately.

The import/not import behavior also clarifies some constraints on sys.path, while scripts do not require any special sys.path, regular tests generally can only be imported from a specific entry point. Since this cannot be guessed reliably, the user has to set it up properly before calling sst-run.

'shared' on the other hand still add itself to sys.path, we may want to revisit that too.

The sys.path requirements (surprisingly) triggered some more cleanups in the existing code where absolute paths were used instead of relative ones.

I've added a bunch of unit tests that should make it easier to refactor the test loader and its companion classes and some helpers that make it easier to test it.

I've also changed the test ids to reflect their path as a pre-requisite to filtering (we weren't doing that previously leading to duplicate test ids if a script was existing under the same base name in different directories.

I did some sprint cleanup on my way which explain partly the size of the proposal.

There are now 97 unit tests (vs the initial ~35) which should help ;) So I'm pretty confident that the introduced features are not too bogus. But since I rewrote code that didn't have corresponding tests, I may have break support for obscure use cases. By contrast, removing get_suites and friends was easier with the existing test_runtests_find_cases.py and test_runtests_get_suites.py.

I've put some aliases in runtests for the newly introduced 'case' and 'browsers' modules. We should decide how to publicize the new layout and how we want to deprecate these aliases.

This proposal is already quite big and while I left some items in the test-loader-TODO file, the bulk of it is ready for feedback.

Warning for reviewers: things may be clearer if review locally and inspect each commit in isolation.

[lp:~vila/selenium-simple-test/test-loader]

435. By Vincent Ladeuil on 2013-05-09

Merge trunk resolving conflicts

436. By Vincent Ladeuil on 2013-05-10

Merge trunk

437. By Vincent Ladeuil on 2013-05-10

Merge fix for run_django

438. By Vincent Ladeuil on 2013-05-10

Merge fixes for result

439. By Vincent Ladeuil on 2013-05-10

Ensures that tests defined in the __init__ file are properly loaded.

440. By Vincent Ladeuil on 2013-05-13

Don't import variables from modules.

441. By Vincent Ladeuil on 2013-05-13

Simpler Loaders implementation.

442. By Vincent Ladeuil on 2013-05-13

Split filtering from loader.py into its own filters.py module, provide tested helpers to simplify runtests.

443. By Vincent Ladeuil on 2013-05-13

Define and use helpers that make it easier to define discover in __init__.py files. Change the discover signature to match the real needs. Update the TODO. All test ids now include the python path of the file where they are defined

[Leo Arias (elopio)]

l 44, 66, 355, 482, 504, 644, 664, 685, 731, 765, 806, 835, 890, 938, 948, 959, 995, 1961, 1981, 2022, 3958
Pep8 recommeds to precede the quotes that end a multiline docstring with an empty line.
Not a real issue, and not all of them introduced by you. But I find nice the consistency that comes from following all pep8 recommendations.

I have on my TODO to update all the comments in the actions module, btw.

139 === added file 'src/sst/case.py'
Isn't cases a better name?

332 + # TODO: Adding script_dir to sys.path only make sense if we want to
333 + # allow scripts to import from their own dir. Do we really need that ?

Typo: makeS

To answer the question, I'd say no. We are trying to use the full path everywhere, so this only allows a behavior we are trying to avoid. If we remove it, it might break some tests, but that's fine as they should be updated anyway.

353 + the first row (headers) match data_map key names.
354 + rows beneath are filled with data values.

Not changed by you, but I think this has wrong identation.

to be continued...

[Leo Arias (elopio)]

726 + """Load tests for a tree containing scrits.

Typo: scriPts

736 + regexp = '^shared$|^_'

The name of the shared file can be parameterized with the -m command line option. Having it hard-coded seems wrong.

752 + # FIXME

Can you file a bug for that? Seems nice to have, not so hard.

787 + # FIXME: This swallows exceptions raise the by the user defined

Typos: raiseD _ by

788 + # 'load_test'. We may want to give awy to expose them instead (with

Typo: a _ wAy

822 + """Load test from an sst tree.

Typo: testS

829 + This also provide ways for packages to define the test loading as they see

Typo: provideS

1886 +assert (config.shared_directory
1887 + == os.path.abspath(os.path.join(thisdir, 'shared')))

According to pep8, the break should be after the operator.

2020 + :param description: A text where files and directories contents is

I'm curious about where did you get this style for documenting parameters. The example from http://www.python.org/dev/peps/pep-0257/#multi-line-docstrings is different, but I'm not sure if there's a "right" or recommended way for this. Personally, I prefer the style from 257, but I can give you no reasons for that :)

(I'm not done yet...)

[Leo Arias (elopio)]

2084 + # command.get_opts_run and friends relies on optparse defaulting so

I think that's a type. Maybe it should be s/so/to

2085 + # sys.argv[1:]. To omply with that, we add a dummy first arg to

Typo: *c*omply

2212 + def assertFiltered(self, expected, condition, actual):

I'm not sure 'actual' is a good name here. When I first read it, I thought expected would be asserted as equal to actual.
So, maybe, call it 'tests'? Not sure if it's better though.
Also, the calls to this method are hard to read. I think it might be better to use keyword args on the tests that call it:

self.assertFiltered(
    expected=['foo', 'foobar', 'barfoo'], pattern=['*foo*'],
    tests=['foo', 'foobar', 'barfoo', 'baz'])

I'm not sure if that's better, either :) Just a thought.

2342 +Because the tests themselves shares this module name space, care must be taken

Typo: tests (...) share_

2424 +class TestModuleLoader

You are missing the test for a file with an _ in front, right?

l. 2450 With textwrap.dedent, this strings might look prettier.

2607 + f.write("'foo'^'bar'\n")

Instead of calling write three times, I would prefer to define a variable csv_contents, and write it once.

I'm almost done...

Awesome job man! Makes me think that a lot of this loaders will be useful to hordes of testers around the world. All except the script loaders may be a good addition to testtools.

I'll finish tomorrow morning. I'm testing this on lp:~elopio/canonical-identity-provider/fix1175698-ubuntu_title that failed because of the wrong original discovery we had. It now finds the cases I'm interested in, but failes for other reasons. So I need to first get SSO to green, to compare. I'll look for someone to blame tomorrow too.

[lp:~vila/selenium-simple-test/test-loader]

444. By Vincent Ladeuil on 2013-05-14

Fix typos mentioned in review.

445. By Vincent Ladeuil on 2013-05-14

Rename sst/case.py to sst/cases.py as mentioned in review and fix fallouts.

[Vincent Ladeuil (vila)]

> l 44, 66, 355, 482, 504, 644, 664, 685, 731, 765, 806, 835, 890, 938, 948,
> 959, 995, 1961, 1981, 2022, 3958
> Pep8 recommeds to precede the quotes that end a multiline docstring with an
> empty line.

But the rationale given is:

  The BDFL [3] recommends inserting a blank line between the last paragraph
  in a multi-line docstring and its closing quotes, placing the closing
  quotes on a line by themselves. This way, Emacs' fill-paragraph command
  can be used on it.

That may have been true in the past but nowadays, fill-paragraph works
perfectly fine there.

I have several other reasons to disagree:

- this rule is not enforced by our pep8 conformance test so it's unlikely to
  be used consistently,

- when generating doc, this extra newline may conflict with other layout
  style and will just complicate matters if we need to special case the end
  of string,

- it wastes space, the triple quote on its own line already provides a
  visual break.

That being said, if you object strongly I'll fix it and will try to respect
it in the future but I think it's far less important than respecting that
rule that prescribes a single line summary at the start of the docstring.

> Not a real issue, and not all of them introduced by you. But I find nice the
> consistency that comes from following all pep8 recommendations.

>
> I have on my TODO to update all the comments in the actions module, btw.

Yeah, that's hhtp://pad.lv/1170389

>
> 139 === added file 'src/sst/case.py'
> Isn't cases a better name?
>

I almost used it myself, fixed. I've found that using the plural form for
modules leads to less clashes with variable names but it's not an idiom that
I encounter often, so I've used it lightly so far.

On the topic of module and class names, should we take the opportunity to
get rid of the redundancy in the test case class names, sst means Selenium
Simple Test, so SSTTestCase means Selenium Simple Test Test Case and
SSTScriptTestCase means Selenium Simple Test Script Test Case. I think it
should be:

- SSTestCase: Selenium Simple Test Case
- SSTScriptCase: Selenium Simple Test Script Case

Since we will deprecate runtests.SSTTestCase and runtests.SSTScriptTestCase
at some point in the future, we may as well fix this so people only have to
update once.

What do you think ?

> 332 + # TODO: Adding script_dir to sys.path only make sense if we want to
> 333 + # allow scripts to import from their own dir. Do we really need that
> ?
>
> Typo: makeS
>
> To answer the question, I'd say no. We are trying to use the full path
> everywhere, so this only allows a behavior we are trying to avoid. If we
> remove it, it might break some tests, but that's fine as they should be
> updated anyway.

Ok, filed http://pad.lv/1179797

>
> 353 + the first row (headers) match data_map key names.
> 354 + rows beneath are filled with data values.
>
> Not changed by you, but I think this has wrong identation.

Fixed. Using fill-paragraph even ;)

>
> to be continued...

Thanks, very much appreciated.

[lp:~vila/selenium-simple-test/test-loader]

446. By Vincent Ladeuil on 2013-05-14

More typos mentioned in review.

447. By Vincent Ladeuil on 2013-05-14

More typos and other test refactorings mentioned in review.

[Vincent Ladeuil (vila)]

> 2084 + # command.get_opts_run and friends relies on optparse defaulting so
>
> I think that's a type. Maybe it should be s/so/to

Tyop indeed, fixed.

>
> 2085 + # sys.argv[1:]. To omply with that, we add a dummy first arg to
>
> Typo: *c*omply
>

Fixed.

> 2212 + def assertFiltered(self, expected, condition, actual):
>
> I'm not sure 'actual' is a good name here. When I first read it, I thought
> expected would be asserted as equal to actual.
> So, maybe, call it 'tests'? Not sure if it's better though.

You're right, I renamed it 'ids as tests are created from them, not
received as such and not compared to expected indeed.

> Also, the calls to this method are hard to read. I think it might be better to
> use keyword args on the tests that call it:
>
> self.assertFiltered(
> expected=['foo', 'foobar', 'barfoo'], pattern=['*foo*'],
> tests=['foo', 'foobar', 'barfoo', 'baz'])
>
> I'm not sure if that's better, either :) Just a thought.

Yeah, I don't think adding keywords here is worth the added noise. You need
to understand what the helper is doing in any case, the keywords won't give
you that knowledge. Each class is short enough that you can afford reading
the helper to understand single line tests IMHO.

I've added docstrings to the assertFiltered methods.

>
> 2342 +Because the tests themselves shares this module name space, care must
> be taken
>
> Typo: tests (...) share_
>

Fixed.

> 2424 +class TestModuleLoader
>
> You are missing the test for a file with an _ in front, right?

test_ignore_privates ?

>
> l. 2450 With textwrap.dedent, this strings might look prettier.
>

I had several discussions about that in the past on various projects.

While relying on dedent seems a good idea at first, when maintaining the
tests it proves to be more trouble than the initial supposed easier read:

- the added indentations are not always appropriate as it means the input
  you type transformed before being used so you're not looking at the data
  your test will process,

- this makes it hard to copy paste code or data you tuned somewhere else
  (variation on the point above: better stick with the format that is
  processed).

> 2607 + f.write("'foo'^'bar'\n")
>
> Instead of calling write three times, I would prefer to define a variable
> csv_contents, and write it once.

Hehe, perfect example that I should have followed my own advices
above. Rewriting this as a multi-line content is far clearer (some single
quotes are required there and it's easier to catch them while reading a
single description than three different lines using different quotes).

It's also clearly a case where I prefer to respect the format expected than
adding leading indentation.

>
> I'm almost done...
>
> Awesome job man! Makes me think that a lot of this loaders will be useful to
> hordes of testers around the world. All except the script loaders may be a
> good addition to testtools.

Yeah, that was more or less the idea, first make it works in sst then look
at what can be upstreamed to testtools and/or python, but there is no hurry
;) sst really introduces an unusual pattern because the scripts should not
be imported, a constra...

[Vincent Ladeuil (vila)]

Dang, my answer to that has been lost :-(

> 726 + """Load tests for a tree containing scrits.
>
> Typo: scriPts

Fixed.

>
> 736 + regexp = '^shared$|^_'
>
> The name of the shared file can be parameterized with the -m command line
> option. Having it hard-coded seems wrong.

It was hard-coded in different places including:

- find_shared_directory which will search directories above for a 'shared'
  dir when -m is not used,

- in get_suite and friends functions I'm replacing.

So it should be ignored for compatibility.

Now, I think '-m' is just a convenience to add a directory to sys.path and
I'm not convinced it's better than letting the user properly setup
PYTHONPATH or sys.path before running sst-run. So my plan is to remove
find_shared_directory and just leave the '-m' in place for now.

>
> 752 + # FIXME
>
> Can you file a bug for that? Seems nice to have, not so hard.

That's part of the refactoring I plan to do so should be fixed in the final
proposal.

>
> 787 + # FIXME: This swallows exceptions raise the by the user defined
>
> Typos: raiseD _ by
>

Fixed.

> 788 + # 'load_test'. We may want to give awy to expose them instead (with
>
> Typo: a _ wAy
>

Fixed.

> 822 + """Load test from an sst tree.
>
> Typo: testS
>

Fixed.

> 829 + This also provide ways for packages to define the test loading as
> they see
>
> Typo: provideS
>

Fixed.

> 1886 +assert (config.shared_directory
> 1887 + == os.path.abspath(os.path.join(thisdir, 'shared')))
>

Rewritten to avoid the issue.

> According to pep8, the break should be after the operator.
>

I disagree with that, the operator is the important part, it's easier to
find at the beginning of the line than at the end.

And again, our pep8 test didn't catch it...

> 2020 + :param description: A text where files and directories contents is
>
> I'm curious about where did you get this style for documenting parameters. The
> example from http://www.python.org/dev/peps/pep-0257/#multi-line-docstrings is
> different, but I'm not sure if there's a "right" or recommended way for this.
> Personally, I prefer the style from 257, but I can give you no reasons for
> that :)
>

I think adding markup here doesn't make it harder to read but pave the way
for better doc generation in the future.

Some references:

  http://stackoverflow.com/questions/5334531/python-documentation-standard-for-docstring
  http://sphinx-doc.org/markup/desc.html#info-field-lists

> (I'm not done yet...)

Thanks for that already ;)

[lp:~vila/selenium-simple-test/test-loader]

448. By Vincent Ladeuil on 2013-05-14

Stop adding the script directory to sys.path

[Leo Arias (elopio)]

> That being said, if you object strongly I'll fix it and will try to respect
> it in the future but I think it's far less important than respecting that
> rule that prescribes a single line summary at the start of the docstring.

I'm not objecting strongly, you have better reasons than I do.

> On the topic of module and class names, should we take the opportunity to
> get rid of the redundancy in the test case class names, sst means Selenium
> Simple Test, so SSTTestCase means Selenium Simple Test Test Case and
> SSTScriptTestCase means Selenium Simple Test Script Test Case. I think it
> should be:
>
> - SSTestCase: Selenium Simple Test Case
> - SSTScriptCase: Selenium Simple Test Script Case
>
> Since we will deprecate runtests.SSTTestCase and runtests.SSTScriptTestCase
> at some point in the future, we may as well fix this so people only have to
> update once.
>
> What do you think ?

I think we should do both at the same time. I can take care of updating sso, pay and u1, it's not hard.
However, that was released on a pypi, might break tests from other users of SST. I'm not sure if there are lots of them, we probably should ask Corey.

[Leo Arias (elopio)]

> > Also, the calls to this method are hard to read. I think it might be better
> to
> > use keyword args on the tests that call it:
> >
> > self.assertFiltered(
> > expected=['foo', 'foobar', 'barfoo'], pattern=['*foo*'],
> > tests=['foo', 'foobar', 'barfoo', 'baz'])
> >
> > I'm not sure if that's better, either :) Just a thought.
>
> Yeah, I don't think adding keywords here is worth the added noise. You need
> to understand what the helper is doing in any case, the keywords won't give
> you that knowledge. Each class is short enough that you can afford reading
> the helper to understand single line tests IMHO.
>
> I've added docstrings to the assertFiltered methods.

That's good enough. Thanks.

> > 2424 +class TestModuleLoader
> >
> > You are missing the test for a file with an _ in front, right?
>
> test_ignore_privates ?

That one is in TestScriptLoader. If you change the implementation of script loader to not use ignore privates from the module loader, you can end with a coverage hole.

[Vincent Ladeuil (vila)]

> > Since we will deprecate runtests.SSTTestCase and runtests.SSTScriptTestCase
> > at some point in the future, we may as well fix this so people only have to
> > update once.
> >
> > What do you think ?
>
> I think we should do both at the same time. I can take care of updating sso,
> pay and u1, it's not hard.
> However, that was released on a pypi, might break tests from other users of
> SST. I'm not sure if there are lots of them, we probably should ask Corey.

To clarify: I've left the symbols defined in runtests.py so we won't break compatibility by landing this.

The day we want to deprecate them, we still won't break compatibility but emit warnings.

So, say, 0.2.4 will introduce the refactor but the old symbols are still available.
0.2.5 will emit warnings about the symbols being deprecated and pointing to the new ones.
0.2.6 will remove the old symbols.

Or we can just emit the warnings now but I'd rather do that in a different proposal.

[Leo Arias (elopio)]

again, this is awesome.
I'll leave it to you to globally approve. If you agree with me about the missing test, then you can add it. If you don't agree, then don't :)

review: Approve (code review)

[lp:~vila/selenium-simple-test/test-loader]

449. By Vincent Ladeuil on 2013-05-14

Remerge result branch

=== modified file 'docs/changelog.rst'
--- docs/changelog.rst	2013-05-10 08:53:33 +0000
+++ docs/changelog.rst	2013-05-14 11:43:29 +0000
@@ -24,6 +24,7 @@
 * switched to `junitxml` dependency for XML report generation.
 * refactored ``retry_on_stale_element`` to make a new more generic
   ``retry_on_exception``.
+* the script directory is not added to sys.path implicitly anymore.
 
 
 version **0.2.3** (2013 Apr 17)

=== modified file 'src/sst/actions.py'
--- src/sst/actions.py	2013-05-10 08:53:33 +0000
+++ src/sst/actions.py	2013-05-14 11:43:29 +0000
@@ -246,6 +246,7 @@
     try:
         os.makedirs(config.results_directory)
     except OSError:
+        # FIXME: We should only catch the EEXIST errno -- vila 2013-04-29
         pass  # already exists
 
 

=== added file 'src/sst/browsers.py'
--- src/sst/browsers.py	1970-01-01 00:00:00 +0000
+++ src/sst/browsers.py	2013-05-14 11:43:29 +0000
@@ -0,0 +1,121 @@
+#
+#   Copyright (c) 2011-2013 Canonical Ltd.
+#
+#   This file is part of: SST (selenium-simple-test)
+#   https://launchpad.net/selenium-simple-test
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+
+
+from selenium import webdriver
+
+
+class BrowserFactory(object):
+    """Handle browser creation for tests.
+
+    One instance is used for a given test run.
+    """
+
+    webdriver_class = None
+
+    def __init__(self, javascript_disabled=False):
+        super(BrowserFactory, self).__init__()
+        self.javascript_disabled = javascript_disabled
+
+    def setup_for_test(self, test):
+        """Setup the browser for the given test.
+
+        Some browsers accept more options that are test (and browser) specific.
+
+        Daughter classes should redefine this method to capture them.
+        """
+        pass
+
+    def browser(self):
+        """Create a browser based on previously collected options.
+
+        Daughter classes should override this method if they need to provide
+        more context.
+        """
+        return self.webdriver_class()
+
+
+# MISSINGTEST: Exercise this class -- vila 2013-04-11
+class RemoteBrowserFactory(BrowserFactory):
+
+    webdriver_class = webdriver.Remote
+
+    def __init__(self, capabilities, remote_url):
+        super(RemoteBrowserFactory, self).__init__()
+        self.capabilities = capabilities
+        self.remote_url = remote_url
+
+    def browser(self):
+        return self.webdriver_class(self.capabilities, self.remote_url)
+
+
+# MISSINGTEST: Exercise this class -- vila 2013-04-11
+class ChromeFactory(BrowserFactory):
+
+    webdriver_class = webdriver.Chrome
+
+
+# MISSINGTEST: Exercise this class (requires windows) -- vila 2013-04-11
+class IeFactory(BrowserFactory):
+
+    webdriver_class = webdriver.Ie
+
+
+# MISSINGTEST: Exercise this class -- vila 2013-04-11
+class PhantomJSFactory(BrowserFactory):
+
+    webdriver_class = webdriver.PhantomJS
+
+
+# MISSINGTEST: Exercise this class -- vila 2013-04-11
+class OperaFactory(BrowserFactory):
+
+    webdriver_class = webdriver.Opera
+
+
+class FirefoxFactory(BrowserFactory):
+
+    webdriver_class = webdriver.Firefox
+
+    def setup_for_test(self, test):
+        profile = webdriver.FirefoxProfile()
+        profile.set_preference('intl.accept_languages', 'en')
+        if test.assume_trusted_cert_issuer:
+            profile.set_preference('webdriver_assume_untrusted_issuer', False)
+            profile.set_preference(
+                'capability.policy.default.Window.QueryInterface', 'allAccess')
+            profile.set_preference(
+                'capability.policy.default.Window.frameElement.get',
+                'allAccess')
+        if test.javascript_disabled or self.javascript_disabled:
+            profile.set_preference('javascript.enabled', False)
+        self.profile = profile
+
+    def browser(self):
+        return self.webdriver_class(self.profile)
+
+
+# MISSINGTEST: Exercise this class -- vila 2013-04-11
+browser_factories = {
+    'Chrome': ChromeFactory,
+    'Firefox': FirefoxFactory,
+    'Ie': IeFactory,
+    'Opera': OperaFactory,
+    'PhantomJS': PhantomJSFactory,
+}

=== added file 'src/sst/cases.py'
--- src/sst/cases.py	1970-01-01 00:00:00 +0000
+++ src/sst/cases.py	2013-05-14 11:43:29 +0000
@@ -0,0 +1,233 @@
+#
+#   Copyright (c) 2011-2013 Canonical Ltd.
+#
+#   This file is part of: SST (selenium-simple-test)
+#   https://launchpad.net/selenium-simple-test
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+
+import ast
+import logging
+import os
+import pdb
+import sys
+import testtools
+import testtools.content
+
+
+from sst import (
+    actions,
+    browsers,
+    config,
+    context,
+    xvfbdisplay,
+)
+import traceback
+
+
+logger = logging.getLogger('SST')
+
+
+class SSTTestCase(testtools.TestCase):
+    """A test case that can use the sst framework."""
+
+    xvfb = None
+    xserver_headless = False
+
+    browser_factory = browsers.FirefoxFactory()
+
+    javascript_disabled = False
+    assume_trusted_cert_issuer = False
+
+    wait_timeout = 10
+    wait_poll = 0.1
+    base_url = None
+
+    results_directory = os.path.abspath('results')
+    screenshots_on = False
+    debug_post_mortem = False
+    extended_report = False
+
+    def setUp(self):
+        super(SSTTestCase, self).setUp()
+        if self.base_url is not None:
+            actions.set_base_url(self.base_url)
+        actions._set_wait_timeout(self.wait_timeout, self.wait_poll)
+        # Ensures sst.actions will find me
+        actions._test = self
+        if self.xserver_headless and self.xvfb is None:
+            # If we need to run headless and no xvfb is already running, start
+            # a new one for the current test, scheduling the shutdown for the
+            # end of the test.
+            self.xvfb = xvfbdisplay.use_xvfb_server(self)
+        config.results_directory = self.results_directory
+        actions._make_results_dir()
+        self.start_browser()
+        self.addCleanup(self.stop_browser)
+        if self.screenshots_on:
+            self.addOnException(self.take_screenshot_and_page_dump)
+        if self.debug_post_mortem:
+            self.addOnException(
+                self.print_exception_and_enter_post_mortem)
+        if self.extended_report:
+            self.addOnException(self.report_extensively)
+
+    def shortDescription(self):
+        # testools wrongly defines this as returning self.id(). Since we're not
+        # using the short description (aka the first line of the test
+        # docstring) (who is ? should we ?), we revert to the default behavior
+        # so runners and results don't get mad.
+        return None
+
+    def start_browser(self):
+        logger.debug('\nStarting browser')
+        self.browser_factory.setup_for_test(self)
+        self.browser = self.browser_factory.browser()
+        logger.debug('Browser started: %s' % (self.browser.name))
+
+    def stop_browser(self):
+        logger.debug('Stopping browser')
+        self.browser.quit()
+
+    def take_screenshot_and_page_dump(self, exc_info):
+        try:
+            filename = 'screenshot-{0}.png'.format(self.id())
+            actions.take_screenshot(filename)
+        except Exception:
+            # FIXME: Needs to be reported somehow ? -- vila 2012-10-16
+            pass
+        try:
+            # also dump page source
+            filename = 'pagesource-{0}.html'.format(self.id())
+            actions.save_page_source(filename)
+        except Exception:
+            # FIXME: Needs to be reported somehow ? -- vila 2012-10-16
+            pass
+
+    def print_exception_and_enter_post_mortem(self, exc_info):
+        exc_class, exc, tb = exc_info
+        traceback.print_exception(exc_class, exc, tb)
+        pdb.post_mortem(tb)
+
+    def report_extensively(self, exc_info):
+        exc_class, exc, tb = exc_info
+        original_message = str(exc)
+        try:
+            current_url = actions.get_current_url()
+        except Exception:
+            current_url = 'unavailable'
+        try:
+            page_source = actions.get_page_source()
+        except Exception:
+            page_source = 'unavailable'
+        self.addDetail(
+            'Original exception',
+            testtools.content.text_content('{0} : {1}'.format(
+                exc.__class__.__name__, original_message)))
+        self.addDetail('Current url',
+                       testtools.content.text_content(current_url))
+        self.addDetail('Page source',
+                       testtools.content.text_content(page_source))
+
+
+class SSTScriptTestCase(SSTTestCase):
+    """Test case used internally by sst-run and sst-remote."""
+
+    def __init__(self, script_dir, script_name, context_row=None):
+        super(SSTScriptTestCase, self).__init__('run_test_script')
+        self.script_dir = script_dir
+        self.script_name = script_name
+        self.script_path = os.path.join(self.script_dir, self.script_name)
+
+        # pythonify the script path into a python path
+        test_id = self.script_path.replace('.py', '')
+        if test_id.startswith('./'):
+            test_id = test_id[2:]
+        self.id = lambda: '%s' % (test_id.replace(os.sep, '.'))
+        if context_row is None:
+            context_row = {}
+        self.context = context_row
+
+    def __str__(self):
+        # Since we use run_test_script to encapsulate the call to the
+        # compiled code, we need to override __str__ to get a proper name
+        # reported.
+        return "%s" % (self.id(),)
+
+    def setUp(self):
+        self._compile_script()
+        # The script may override some settings. The default value for
+        # JAVASCRIPT_DISABLED and ASSUME_TRUSTED_CERT_ISSUER are False, so if
+        # the user mentions them in his script, it's to turn them on. Also,
+        # getting our hands on the values used in the script is too hackish ;)
+        if 'JAVASCRIPT_DISABLED' in self.code.co_names:
+            self.javascript_disabled = True
+        if 'ASSUME_TRUSTED_CERT_ISSUER' in self.code.co_names:
+            self.assume_trusted_cert_issuer = True
+        super(SSTScriptTestCase, self).setUp()
+        # Start with default values
+        actions.reset_base_url()
+        actions._set_wait_timeout(10, 0.1)
+        # Possibly inject parametrization from associated .csv file
+        previous_context = context.store_context()
+        self.addCleanup(context.restore_context, previous_context)
+        context.populate_context(self.context, self.script_path,
+                                 self.browser.name, self.javascript_disabled)
+
+    def _compile_script(self):
+        self.script_path = os.path.join(self.script_dir, self.script_name)
+        with open(self.script_path) as f:
+            source = f.read() + '\n'
+        self.code = compile(source, self.script_path, 'exec')
+
+    def run_test_script(self, result=None):
+        # Run the test catching exceptions sstnam style
+        try:
+            exec self.code in self.context
+        except actions.EndTest:
+            pass
+
+
+def get_data(csv_path):
+    """
+    Return a list of data dicts for parameterized testing.
+
+    The first row (headers) match data_map key names. rows beneath are filled
+    with data values.
+    """
+    rows = []
+    print '  Reading data from %r...' % os.path.split(csv_path)[-1],
+    row_num = 0
+    with open(csv_path) as f:
+        headers = f.readline().rstrip().split('^')
+        headers = [header.replace('"', '') for header in headers]
+        headers = [header.replace("'", '') for header in headers]
+        for line in f:
+            row = {}
+            row_num += 1
+            row['_row_num'] = row_num
+            fields = line.rstrip().split('^')
+            for header, field in zip(headers, fields):
+                try:
+                    value = ast.literal_eval(field)
+                except ValueError:
+                    value = field
+                    if value.lower() == 'false':
+                        value = False
+                    if value.lower() == 'true':
+                        value = True
+                row[header] = value
+            rows.append(row)
+    print 'found %s rows' % len(rows)
+    return rows

=== modified file 'src/sst/command.py'
--- src/sst/command.py	2013-05-01 22:21:07 +0000
+++ src/sst/command.py	2013-05-14 11:43:29 +0000
@@ -28,6 +28,7 @@
 import sst
 from sst import (
     actions,
+    browsers,
     config,
     runtests,
 )
@@ -95,6 +96,13 @@
     parser.add_option('--collect-only', dest='collect_only',
                       action='store_true', default=False,
                       help='collect/print cases without running tests')
+    parser.add_option('-i', '--include', dest='includes',
+                      action='append',
+                      help='all tests starting with this prefix will be run')
+    parser.add_option(
+        '-e', '--exclude', dest='excludes',
+        action='append',
+        help='all tests starting with this prefix will not be run')
     return parser
 
 
@@ -132,17 +140,17 @@
     return parser
 
 
-def get_opts_run():
-    return get_opts(get_run_options)
-
-
-def get_opts_remote():
-    return get_opts(get_remote_options)
-
-
-def get_opts(get_options):
+def get_opts_run(args=None):
+    return get_opts(get_run_options, args)
+
+
+def get_opts_remote(args=None):
+    return get_opts(get_remote_options, args)
+
+
+def get_opts(get_options, args=None):
     parser = get_options()
-    (cmd_opts, args) = parser.parse_args()
+    (cmd_opts, args) = parser.parse_args(args)
 
     if cmd_opts.print_version:
         print 'SST version: %s' % sst.__version__
@@ -156,7 +164,7 @@
         print 'run "%s -h" or "%s --help" to see run options.' % (prog, prog)
         sys.exit(1)
 
-    if cmd_opts.browser_type not in runtests.browser_factories:
+    if cmd_opts.browser_type not in browsers.browser_factories:
         print ("Error: %s should be one of %s"
                % (cmd_opts.browser_type, runtests.browser_factories.keys()))
         sys.exit(1)

=== added file 'src/sst/filters.py'
--- src/sst/filters.py	1970-01-01 00:00:00 +0000
+++ src/sst/filters.py	2013-05-14 11:43:29 +0000
@@ -0,0 +1,97 @@
+#
+#   Copyright (c) 2013 Canonical Ltd.
+#
+#   This file is part of: SST (selenium-simple-test)
+#   https://launchpad.net/selenium-simple-test
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+import fnmatch
+import unittest
+
+
+def filter_suite(condition, suite):
+    """Return tests for which ``condition`` is True in ``suite``.
+
+    :param condition: A callable receiving a test and returning True if the
+        test should be kept.
+
+    :param suite: A test suite that can be iterated. It contains either tests
+        or suite inheriting from ``unittest.TestSuite``.
+
+    ``suite`` is a tree of tests and suites, the returned suite respect the
+    received suite layout, only removing empty suites.
+    """
+    filtered_suite = suite.__class__()
+    for test in suite:
+        if issubclass(test.__class__, unittest.TestSuite):
+            # We received a suite, we'll filter a suite
+            filtered = filter_suite(condition, test)
+            if filtered.countTestCases():
+                # Keep only non-empty suites
+                filtered_suite.addTest(filtered)
+        elif condition(test):
+            # The test is kept
+            filtered_suite.addTest(test)
+    return filtered_suite
+
+
+def filter_by_patterns(patterns, suite):
+    """Returns the tests that match one of ``patterns``.
+
+    :param patterns: A list of test name globs to include. All tests are
+        included if no patterns are provided.
+
+    :param suite: The test suite to filter.
+    """
+    if not patterns:
+        return suite
+
+    def filter_test_patterns(test):
+        for pattern in patterns:
+            if fnmatch.fnmatchcase(test.id(), pattern):
+                return True
+        return False
+    return filter_suite(filter_test_patterns, suite)
+
+
+def include_prefixes(prefixes, suite):
+    """Returns the tests whose id starts with one of the prefixes."""
+    if not prefixes:
+        # No prefixes, no filtering
+        return suite
+
+    def starts_with_one_of(test):
+        # A test is kept if its id starts with one of the prefixes
+        tid = test.id()
+        for prefix in prefixes:
+            if tid.startswith(prefix):
+                return True
+        return False
+    return filter_suite(starts_with_one_of, suite)
+
+
+def exclude_prefixes(prefixes, suite):
+    """Returns the tests whose id does not start with any of the prefixes."""
+    if not prefixes:
+        # No prefixes, no filtering
+        return suite
+
+    def starts_with_none_of(test):
+        # A test is kept if its id matches none of the 'excludes' prefixes
+        tid = test.id()
+        for prefix in prefixes:
+            if tid.startswith(prefix):
+                return False
+        return True
+    return filter_suite(starts_with_none_of, suite)

=== added file 'src/sst/loader.py'
--- src/sst/loader.py	1970-01-01 00:00:00 +0000
+++ src/sst/loader.py	2013-05-14 11:43:29 +0000
@@ -0,0 +1,409 @@
+#
+#   Copyright (c) 2011-2013 Canonical Ltd.
+#
+#   This file is part of: SST (selenium-simple-test)
+#   https://launchpad.net/selenium-simple-test
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+import contextlib
+import fnmatch
+import functools
+import os
+import re
+import sys
+import unittest
+import unittest.loader
+
+from sst import cases
+
+
+def matches_for_regexp(regexp):
+    match_re = re.compile(regexp)
+
+    def matches(path):
+        return bool(match_re.match(path))
+    return matches
+
+
+def matches_for_glob(pattern):
+    match_re = fnmatch.translate(pattern)
+    return matches_for_regexp(match_re)
+
+
+class NameMatcher(object):
+
+    def __init__(self, includes=None, excludes=None):
+        if includes is not None:
+            self.includes = includes
+        if excludes is not None:
+            self.excludes = excludes
+
+    def includes(self, path):
+        return True
+
+    def excludes(self, path):
+        return False
+
+    def matches(self, name):
+        return self.includes(name) and not self.excludes(name)
+
+
+class FileLoader(object):
+    """Load tests from a file.
+
+    This is an abstract class allowing daughter classes to enforce constraints
+    including the ability to load tests from files that cannot be imported.
+    """
+
+    def __init__(self, test_loader, matcher=None):
+        """Load tests from a file."""
+        super(FileLoader, self).__init__()
+        if matcher is None:
+            self.matches = lambda name: True
+        else:
+            self.matches = matcher.matches
+        self.test_loader = test_loader
+
+    def discover(self, directory, name):
+        """Return None to represent an empty test suite.
+
+        This is mostly for documentation purposes, if a file contains material
+        that can produce tests, a specific file loader should be defined to
+        build tests from the file content.
+        """
+        return None
+
+
+class ModuleLoader(FileLoader):
+    """Load tests from a python module.
+
+    This handles base name matching and loading tests defined in an importable
+    python module.
+    """
+
+    def __init__(self, test_loader, matcher=None):
+        if matcher is None:
+            # Default to python source files, excluding private ones
+            matcher = NameMatcher(includes=matches_for_regexp('.*\.py$'),
+                                  excludes=matches_for_regexp('^_'))
+        super(ModuleLoader, self).__init__(test_loader, matcher=matcher)
+
+    def discover(self, directory, name):
+        if not self.matches(name):
+            return None
+        module = self.test_loader.importFromPath(os.path.join(directory, name))
+        return self.test_loader.loadTestsFromModule(module)
+
+
+class ScriptLoader(FileLoader):
+    """Load tests from an sst script.
+
+    This handles base name matching and loading tests defined in an sst script.
+    """
+
+    def __init__(self, test_loader, matcher=None):
+        if matcher is None:
+            # Default to python source files, excluding private ones
+            matcher = NameMatcher(includes=matches_for_regexp('.*\.py$'),
+                                  excludes=matches_for_regexp('^_'))
+        super(ScriptLoader, self).__init__(test_loader, matcher=matcher)
+
+    def discover(self, directory, name):
+        if not self.matches(name):
+            return None
+        return self.test_loader.loadTestsFromScript(directory, name)
+
+
+class DirLoader(object):
+    """Load tests from a tree.
+
+    This is an abstract class allowing daughter classes to enforce constraints
+    including the ability to load tests from files and directories that cannot
+    be imported.
+    """
+
+    def __init__(self, test_loader, matcher=None):
+        """Load tests from a directory."""
+        super(DirLoader, self).__init__()
+        if matcher is None:
+            # Accept everything
+            self.matches = lambda name: True
+        else:
+            self.matches = matcher.matches
+        self.test_loader = test_loader
+
+    def discover(self, directory, name):
+        if not self.matches(name):
+            return None
+        path = os.path.join(directory, name)
+        names = os.listdir(path)
+        names = self.test_loader.sortNames(names)
+        return self.discover_names(path, names)
+
+    def discover_names(self, directory, names):
+        suite = self.test_loader.suiteClass()
+        for name in names:
+            tests = self.discover_path(directory, name)
+            if tests is not None:
+                suite.addTests(tests)
+        return suite
+
+    def discover_path(self, directory, name):
+        loader = None
+        path = os.path.join(directory, name)
+        if os.path.isfile(path):
+            loader = self.test_loader.fileLoaderClass(self.test_loader)
+        elif os.path.isdir(path):
+            loader = self.test_loader.dirLoaderClass(self.test_loader)
+        if loader is not None:
+            return loader.discover(directory, name)
+        return None
+
+
+class ScriptDirLoader(DirLoader):
+    """Load tests for a tree containing scripts.
+
+    Scripts can be organized in a tree where directories are not python
+    packages. Since scripts are not imported, they don't require the
+    directories containing them to be packages.
+    """
+
+    def __init__(self, test_loader, matcher=None):
+        if matcher is None:
+            # Excludes the 'shared' directory and the "private" directories
+            regexp = '^shared$|^_'
+            matcher = NameMatcher(excludes=matches_for_regexp(regexp))
+        super(ScriptDirLoader, self).__init__(test_loader, matcher=matcher)
+
+    def discover_path(self, directory, name):
+        # MISSINGTEST: The behavior is unclear when a module cannot be imported
+        # because sys.path is incomplete. This makes it hard for the user to
+        # understand it should update sys.path -- vila 2013-05-05
+        path = os.path.join(directory, name)
+        if (os.path.isdir(path) and os.path.isfile(
+                os.path.join(path, '__init__.py'))):
+            # Hold on, we need to respect users wishes here (if it has some)
+            loader = PackageLoader(self.test_loader)
+            try:
+                return loader.discover(directory, name)
+            except ImportError:
+                # FIXME: Nah, didn't work, should we report it to the user ?
+                # (yes see MISSINGTEST above) How ? (By re-raising with a
+                # proper message: if there is an __init__.py file here, it
+                # should be importable, that's what we should explain to the
+                # user) vila 2013-05-04
+                pass
+        return super(ScriptDirLoader, self).discover_path(directory, name)
+
+
+class PackageLoader(DirLoader):
+    """Load tests for a package.
+
+    A package provides a way for the user to specify how tests are loaded.
+    """
+
+    def discover(self, directory, name):
+        if not self.matches(name):
+            return None
+        path = os.path.join(directory, name)
+        try:
+            package = self.test_loader.importFromPath(path)
+        except ImportError:
+            # Explicitly raise the full exception with its backtrace. This
+            # could be overwritten by daughter classes to handle them
+            # differently (swallowing included ;)
+            raise
+        # Can we delegate to the package ?
+        discover = getattr(package, 'discover', None)
+        if discover is not None:
+            # Since the user defined it, the package knows better
+            return discover(self.test_loader, package,
+                            os.path.join(directory, name))
+        # Can we use the load_tests protocol ?
+        load_tests = getattr(package, 'load_tests', None)
+        if load_tests is not None:
+            # FIXME: This swallows exceptions raised the by the user defined
+            # 'load_test'. We may want to give a way to expose them instead
+            # (with or without stopping the test loading) -- vila 2013-04-27
+            return self.test_loader.loadTestsFromModule(package)
+        # Anything else with that ?
+        # Nothing for now, thanks
+
+        names = os.listdir(path)
+        names.remove('__init__.py')
+        names = self.test_loader.sortNames(names)
+        return self.discover_names(path, names)
+
+
[e-mail]
+def Loaders(test_loader, file_loader_class, dir_loader_class):
+    """A context manager for loading tests from a tree.
+
+    This is mainly used when walking a tree a requiring a different set of
+    loaders for a subtree.
+    """
+    if file_loader_class is None:
+        file_loader_class = test_loader.fileLoaderClass
+    if dir_loader_class is None:
+        dir_loader_class = test_loader.dirLoaderClass
+    orig = (test_loader.fileLoaderClass, test_loader.dirLoaderClass)
+    try:
+        test_loader.fileLoaderClass = file_loader_class
+        test_loader.dirLoaderClass = dir_loader_class
+        # 'test_loader' will now use the specified file/dir loader classes
+        yield
+    finally:
+        (test_loader.fileLoaderClass, test_loader.dirLoaderClass) = orig
+
+
+class TestLoader(unittest.TestLoader):
+    """Load tests from an sst tree.
+
+    This loader is able to load sst scripts and create test cases with the
+    right sst specific attributes (browser, error handling, reporting).
+
+    This also allows test case based modules to be loaded when appropriate.
+
+    This also provides ways for packages to define the test loading as they see
+    fit.
+
+    Sorting happens on base names inside a directory while walking the tree and
+    on test classes and test method names when loading a module. Those sortings
+    combined provide a test suite where test ids are sorted.
+    """
+
+    dirLoaderClass = PackageLoader
+    fileLoaderClass = ModuleLoader
+
+    def __init__(self, browser_factory=None,
+                 screenshots_on=False, debug_post_mortem=False,
+                 extended_report=False):
+        super(TestLoader, self).__init__()
+        self.browser_factory = browser_factory
+        self.screenshots_on = screenshots_on
+        self.debug_post_mortem = debug_post_mortem
+        self.extended_report = extended_report
+
+    def discover(self, start_dir, pattern='test*.py', top_level_dir=None):
+        if top_level_dir:
+            # For backward compatibility we insert top_level_dir in
+            # sys.path. More complex import rules are left to the caller to
+            # setup properly
+            sys.path.insert(0, top_level_dir)
+
+        class ModuleLoaderFromPattern(ModuleLoader):
+
+            def __init__(self, test_loader):
+                matcher = NameMatcher(includes=matches_for_glob(pattern))
+                super(ModuleLoaderFromPattern, self).__init__(
+                    test_loader, matcher=matcher)
+
+        return self.discoverTests(start_dir,
+                                  file_loader_class=ModuleLoaderFromPattern)
+
+    def discoverTests(self, start_dir, file_loader_class=None,
+                      dir_loader_class=None):
+        with Loaders(self, file_loader_class, dir_loader_class):
+            dir_loader = self.dirLoaderClass(self)
+            return dir_loader.discover(*os.path.split(start_dir))
+
+    def discoverTestsFromPackage(self, package, path, file_loader_class=None,
+                                 dir_loader_class=None):
+        suite = self.suiteClass()
+        suite.addTests(self.loadTestsFromModule(package))
+        names = os.listdir(path)
+        names.remove('__init__.py')
+        names = self.sortNames(names)
+        with Loaders(self, file_loader_class, dir_loader_class):
+            dir_loader = self.dirLoaderClass(self)
+            suite.addTests(dir_loader.discover_names(path, names))
+        return suite
+
+    def sortNames(self, names):
+        """Return 'names' sorted as defined by sortTestMethodsUsing.
+
+        It's a little abuse of sort*TestMethods*Using as we're sorting file
+        names (or even module python paths) but it allows providing a
+        consistent order for the whole suite.
+        """
+        return sorted(names,
+                      key=functools.cmp_to_key(self.sortTestMethodsUsing))
+
+    def importFromPath(self, path):
+        path = os.path.normpath(path)
+        if path.endswith('.py'):
+            path = path[:-3]  # Remove the trailing '.py'
+        mod_name = path.replace(os.path.sep, '.')
+        __import__(mod_name)
+        return sys.modules[mod_name]
+
+    def loadTestsFromScript(self, dir_name, script_name):
+        suite = self.suiteClass()
+        path = os.path.join(dir_name, script_name)
+        if not os.path.isfile(path):
+            return suite
+        # script specific test parametrization
+        csv_path = path.replace('.py', '.csv')
+        if os.path.isfile(csv_path):
+            for row in cases.get_data(csv_path):
+                # row is a dictionary of variables that will magically appear
+                # as globals in the script.
+                test = self.loadTestFromScript(dir_name, script_name, row)
+                suite.addTest(test)
+        else:
+            test = self.loadTestFromScript(dir_name, script_name)
+            suite.addTest(test)
+        return suite
+
+    def loadTestFromScript(self, dir_name, script_name, context=None):
+        test = cases.SSTScriptTestCase(dir_name, script_name, context)
+
+        # FIXME: We shouldn't have to set test attributes manually, something
+        # smells wrong here. -- vila 2013-04-26
+        test.browser_factory = self.browser_factory
+
+        test.screenshots_on = self.screenshots_on
+        test.debug_post_mortem = self.debug_post_mortem
+        test.extended_report = self.extended_report
+
+        return test
+
+
+def discoverTestScripts(test_loader, package, directory):
+    """``discover`` helper to load sst scripts.
+
+    This can be used in a __init__.py file while walking a regular tests tree.
+    """
+    return test_loader.discoverTestsFromPackage(
+        package, directory,
+        file_loader_class=ScriptLoader, dir_loader_class=ScriptDirLoader)
+
+
+def discoverRegularTests(test_loader, package, directory):
+    """``discover`` helper to load regular python files defining tests.
+
+    This can be used in a __init__.py file while walking an sst tests tree.
+    """
+    return test_loader.discoverTestsFromPackage(
+        package, directory,
+        file_loader_class=ModuleLoader, dir_loader_class=PackageLoader)
+
+
+def discoverNoTests(test_loader, *args, **kwargs):
+    """Returns an empty test suite.
+
+    This can be used in a __init__.py file to prune the test loading for a
+    given subtree.
+    """
+    return test_loader.suiteClass()

=== added file 'src/sst/result.py'
--- src/sst/result.py	1970-01-01 00:00:00 +0000
+++ src/sst/result.py	2013-05-14 11:43:29 +0000
@@ -0,0 +1,104 @@
+#
+#   Copyright (c) 2011,2012,2013 Canonical Ltd.
+#
+#   This file is part of: SST (selenium-simple-test)
+#   https://launchpad.net/selenium-simple-test
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+
+import timeit
+
+
+from testtools import testresult
+
+
+class TextTestResult(testresult.TextTestResult):
+    """A TestResult which outputs activity to a text stream.
+
+    TODO: add the verbosity parameter.
+    """
+
+    def __init__(self, stream, failfast=False, verbosity=1, timer=None):
+        super(TextTestResult, self).__init__(stream, failfast)
+        if timer is None:
+            timer = timeit.default_timer
+        self.timer = timer
+        self.verbose = verbosity > 1
+
+    def startTestRun(self):
+        super(TextTestResult, self).startTestRun()
+
+    def startTest(self, test):
+        if self.verbose:
+            self.stream.write(str(test))
+            self.stream.write(' ... ')
+        self.start_time = self.timer()
+        super(TextTestResult, self).startTest(test)
+
+    def stopTest(self, test):
+        if self.verbose:
+            elapsed_time = self.timer() - self.start_time
+            self.stream.write(' (%.3f secs)\n' % elapsed_time)
+            self.stream.flush()
+        super(TextTestResult, self).stopTest(test)
+
+    def addExpectedFailure(self, test, err=None, details=None):
+        if self.verbose:
+            self.stream.write('XFAIL')
+        else:
+            self.stream.write('x')
+        super(TextTestResult, self).addExpectedFailure(test, err, details)
+
+    def addError(self, test, err=None, details=None):
+        if self.verbose:
+            self.stream.write('ERROR')
+        else:
+            self.stream.write('E')
+        super(TextTestResult, self).addError(test, err, details)
+
+    def addFailure(self, test, err=None, details=None):
+        if self.verbose:
+            self.stream.write('FAIL')
+        else:
+            self.stream.write('F')
+        super(TextTestResult, self).addFailure(test, err, details)
+
+    def addSkip(self, test, details=None):
+        # FIXME: Something weird is going on with testtools, as we're supposed
+        # to use a (self, test, reason, details) signature but this is never
+        # called this way -- vila 2013-05-10
+        reason = details.get('reason', '').as_text()
+        if not reason:
+            reason = ''
+        else:
+            reason = ' ' + reason
+        if self.verbose:
+            self.stream.write('SKIP%s' % reason)
+        else:
+            self.stream.write('s')
+        super(TextTestResult, self).addSkip(test, reason, details)
+
+    def addSuccess(self, test, details=None):
+        if self.verbose:
+            self.stream.write('OK')
+        else:
+            self.stream.write('.')
+        super(TextTestResult, self).addSuccess(test, details)
+
+    def addUnexpectedSuccess(self, test, details=None):
+        if self.verbose:
+            self.stream.write('NOTOK')
+        else:
+            self.stream.write('u')
+        super(TextTestResult, self).addUnexpectedSuccess(test, details)

=== modified file 'src/sst/runtests.py'
--- src/sst/runtests.py	2013-05-06 19:19:05 +0000
+++ src/sst/runtests.py	2013-05-14 11:43:29 +0000
@@ -17,34 +17,35 @@
 #   limitations under the License.
 #
 
-import ast
-import fnmatch
 import junitxml
 import logging
 import os
-import pdb
 import sys
-import traceback
-
-from timeit import default_timer
-from unittest import (
-    defaultTestLoader,
-    TestSuite,
-)
-
-from selenium import webdriver
+
+
 import testtools
 import testtools.content
-import testtools.testresult
 from sst import (
     actions,
+    browsers,
+    cases,
     config,
-    context,
-    xvfbdisplay,
-)
-from .actions import (
-    EndTest
-)
+    filters,
+    loader,
+    result,
+)
+
+# Maintaining compatibility until we deprecate the followings
+BrowserFactory = browsers.BrowserFactory
+RemoteBrowserFactory = browsers.RemoteBrowserFactory
+ChromeFactory = browsers.ChromeFactory
+IeFactory = browsers.IeFactory
+PhantomJSFactory = browsers.PhantomJSFactory
+OperaFactory = browsers.OperaFactory
+FirefoxFactory = browsers.FirefoxFactory
+browser_factories = browsers.browser_factories
+SSTTestCase = cases.SSTTestCase
+SSTScriptTestCase = cases.SSTScriptTestCase
 
 
 __all__ = ['runtests']
@@ -52,45 +53,53 @@
 logger = logging.getLogger('SST')
 
 
+# MISSINGTEST: 'shared' relationship with test_dir, auto-added to sys.path or
+# not -- vila 2013-05-05
+# MISSINGTEST: 'results' dir, created in current dir unconditionally conflicts
+# with claim that 'shared' can be found somewhere up -- vila 2013-05-05
 def runtests(test_names, test_dir='.', collect_only=False,
              browser_factory=None,
              report_format='console',
              shared_directory=None, screenshots_on=False, failfast=False,
              debug=False,
-             extended=False):
+             extended=False,
+             includes=None,
+             excludes=None):
+
+    config.results_directory = os.path.abspath('results')
+    actions._make_results_dir()
 
     if test_dir == 'selftests':
         # XXXX horrible hardcoding
         # selftests should be a command instead
         package_dir = os.path.dirname(__file__)
-        test_dir = os.path.join(package_dir, 'selftests')
-
-    test_dir = _get_full_path(test_dir)
-    if not os.path.isdir(test_dir):
-        msg = 'Specified directory %r does not exist' % test_dir
-        print msg
-        sys.exit(1)
+        os.chdir(os.path.dirname(package_dir))
+        test_dir = os.path.join('.', 'sst', 'selftests')
+    else:
+        if not os.path.isdir(test_dir):
+            msg = 'Specified directory %r does not exist' % test_dir
+            print msg
+            sys.exit(1)
+    shared_directory = find_shared_directory(test_dir, shared_directory)
+    config.shared_directory = shared_directory
+    sys.path.append(shared_directory)
 
     if browser_factory is None:
         # TODO: We could raise an error instead as providing a default value
         # makes little sense here -- vila 2013-04-11
-        browser_factory = FirefoxFactory()
-
-    shared_directory = find_shared_directory(test_dir, shared_directory)
-    config.shared_directory = shared_directory
-    sys.path.append(shared_directory)
-
-    config.results_directory = _get_full_path('results')
-
-    test_names = set(test_names)
-
-    suites = get_suites(test_names, test_dir, shared_directory, collect_only,
-                        browser_factory,
-                        screenshots_on, failfast, debug,
-                        extended=extended,
-                        )
-
-    alltests = TestSuite(suites)
+        browser_factory = browsers.FirefoxFactory()
+
+    test_loader = loader.TestLoader(browser_factory, screenshots_on,
+                                    debug, extended)
+    alltests = test_loader.suiteClass()
+    alltests.addTests(
+        test_loader.discoverTests(test_dir,
+                                  file_loader_class=loader.ScriptLoader,
+                                  dir_loader_class=loader.ScriptDirLoader))
+
+    alltests = filters.filter_by_patterns(test_names, alltests)
+    alltests = filters.include_prefixes(includes, alltests)
+    alltests = filters.exclude_prefixes(excludes, alltests)
 
     print ''
     print '  %s test cases loaded\n' % alltests.countTestCases()
@@ -104,46 +113,32 @@
         print 'Collect-Only Enabled, Not Running Tests...\n'
         print 'Tests Collected:'
         print '-' * 16
-        for t in sorted(testtools.testsuite.iterate_tests(alltests)):
+        for t in testtools.testsuite.iterate_tests(alltests):
             print t.id()
-        sys.exit(0)
+        return
 
+    text_result = result.TextTestResult(sys.stdout, failfast=failfast,
+                                        verbosity=2)
     if report_format == 'xml':
-        _make_results_dir()
         results_file = os.path.join(config.results_directory, 'results.xml')
         xml_stream = file(results_file, 'wb')
-        result = testtools.testresult.MultiTestResult(
-            TextTestResult(sys.stdout, failfast=failfast),
+        res = testtools.testresult.MultiTestResult(
+            text_result,
             junitxml.JUnitXmlResult(xml_stream),
         )
-        result.failfast = failfast
+        res.failfast = failfast
     else:
-        result = TextTestResult(sys.stdout, failfast=failfast)
+        res = text_result
 
-    result.startTestRun()
+    res.startTestRun()
     try:
-        alltests.run(result)
+        alltests.run(res)
     except KeyboardInterrupt:
         print >> sys.stderr, 'Test run interrupted'
     finally:
         # XXX should warn on cases that were specified but not found
         pass
-    result.stopTestRun()
-
-
-def _get_full_path(path):
-    return os.path.normpath(
-        os.path.abspath(
-            os.path.join(os.getcwd(), path)
-        )
-    )
-
-
-def _make_results_dir():
-    try:
-        os.makedirs(config.results_directory)
-    except OSError:
-        pass  # already exists
+    res.stopTestRun()
 
 
 def find_shared_directory(test_dir, shared_directory):
@@ -165,9 +160,17 @@
     The intention is that if you have 'tests/shared' and 'tests/foo' you
     run `sst-run -d tests/foo` and 'tests/shared' will still be used as
     the shared directory.
+
+    IMHO the above is only needed because we don't allow:
+    sst-run --start with tests.foo
+
+    So I plan to remove the support for searching 'shared' upwards in favor of
+    allowing running a test subset and go with a sane layout and import
+    behavior. No test fail if this feature is removed so it's not supported
+    anyway. -- vila 2013-04-26
     """
     if shared_directory is not None:
-        return _get_full_path(shared_directory)
+        return os.path.abspath(shared_directory)
 
     cwd = os.getcwd()
     default_shared = os.path.join(test_dir, 'shared')
@@ -180,483 +183,6 @@
                 if os.path.isdir(this_shared):
                     shared_directory = this_shared
                     break
-                relpath = os.path.split(relpath)[0]
-
-    return _get_full_path(shared_directory)
-
-
-def get_suites(test_names, test_dir, shared_dir, collect_only,
-               browser_factory,
-               screenshots_on, failfast, debug,
-               extended=False
-               ):
-    return [
-        get_suite(
-            test_names, root, collect_only,
-            browser_factory,
-            screenshots_on, failfast, debug,
-            extended=extended,
-        )
-        for root, _, _ in os.walk(test_dir, followlinks=True)
-        if os.path.abspath(root) != shared_dir and
-        not os.path.abspath(root).startswith(shared_dir + os.path.sep)
-        and not os.path.split(root)[1].startswith('_')
-    ]
-
-
-def find_cases(test_names, test_dir):
-    found = set()
-    dir_list = os.listdir(test_dir)
-    filtered_dir_list = set()
-    if not test_names:
-        test_names = ['*', ]
-    for name_pattern in test_names:
-        if not name_pattern.endswith('.py'):
-            name_pattern += '.py'
-        matches = fnmatch.filter(dir_list, name_pattern)
-        if matches:
-            for match in matches:
-                if os.path.isfile(os.path.join(test_dir, match)):
-                    filtered_dir_list.add(match)
-    for entry in filtered_dir_list:
-        # conditions for ignoring files
-        if not entry.endswith('.py'):
-            continue
-        if entry.startswith('_'):
-            continue
-        found.add(entry)
-
-    return found
-
-
-def get_suite(test_names, test_dir, collect_only,
-              browser_factory,
-              screenshots_on, failfast, debug,
-              extended=False):
-
-    suite = TestSuite()
-
-    for case in find_cases(test_names, test_dir):
-        csv_path = os.path.join(test_dir, case.replace('.py', '.csv'))
-        if os.path.isfile(csv_path):
-            # reading the csv file now
-            for row in get_data(csv_path):
-                # row is a dictionary of variables
-                suite.addTest(
-                    get_case(
-                        test_dir, case, browser_factory, screenshots_on, row,
-                        failfast=failfast, debug=debug, extended=extended
-                    )
-                )
-        else:
-            suite.addTest(
-                get_case(
-                    test_dir, case, browser_factory, screenshots_on,
-                    failfast=failfast, debug=debug, extended=extended
-                )
-            )
-
-    return suite
-
-
-def use_xvfb_server(test, xvfb=None):
-    """Setup an xvfb server for a given test.
-
-    :param xvfb: An Xvfb object to use. If none is supplied, default values are
-        used to build it.
-
-    :returns: The xvfb server used so tests can use the built one.
-    """
-    if xvfb is None:
-        xvfb = xvfbdisplay.Xvfb()
-    xvfb.start()
-    test.addCleanup(xvfb.stop)
-    return xvfb
-
-
-class TextTestResult(testtools.testresult.TextTestResult):
-    """A TestResult which outputs activity to a text stream.
-
-    TODO: add the verbosity parameter.
-    """
-
-    def __init__(self, stream, failfast=False):
-        super(TextTestResult, self).__init__(stream, failfast)
-
-    def startTestRun(self):
-        super(TextTestResult, self).startTestRun()
-
-    def startTest(self, test):
-        self.stream.write(str(test))
-        self.stream.write(' ...\n')
-        self.start_time = default_timer()
-        super(TextTestResult, self).startTest(test)
-
-    def stopTest(self, test):
-        self.stream.write('\n')
-        self.stream.flush()
-        super(TextTestResult, self).stopTest(test)
-
-    def addExpectedFailure(self, test, err=None, details=None):
-        self.stream.write('Expected Failure\n')
-        super(TextTestResult, self).addExpectedFailure(test, err, details)
-
-    def addError(self, test, err=None, details=None):
-        self.stream.write('ERROR\n')
-        super(TextTestResult, self).addError(test, err, details)
-
-    def addFailure(self, test, err=None, details=None):
-        self.stream.write('FAIL\n')
-        super(TextTestResult, self).addFailure(test, err, details)
-
-    def addSkip(self, test, reason=None, details=None):
-        if reason is None:
-            self.stream.write('Skipped\n')
-        else:
-            self.stream.write('Skipped %r\n' % reason)
-        super(TextTestResult, self).addSkip(test, reason, details)
-
-    def addSuccess(self, test, details=None):
-        elapsed_time = default_timer() - self.start_time
-        self.stream.write('OK (%.3f secs)' % elapsed_time)
-        super(TextTestResult, self).addSuccess(test, details)
-
-    def addUnexpectedSuccess(self, test, details=None):
-        self.stream.write('Unexpected Success\n')
-        super(TextTestResult, self).addUnexpectedSuccess(test, details)
-
-
-class BrowserFactory(object):
-    """Handle browser creation for tests.
-
-    One instance is used for a given test run.
-    """
-
-    webdriver_class = None
-
-    def __init__(self, javascript_disabled=False):
-        super(BrowserFactory, self).__init__()
-        self.javascript_disabled = javascript_disabled
-
-    def setup_for_test(self, test):
-        """Setup the browser for the given test.
-
-        Some browsers accept more options that are test (and browser) specific.
-
-        Daughter classes should redefine this method to capture them.
-        """
-        pass
-
-    def browser(self):
-        """Create a browser based on previously collected options.
-
-        Daughter classes should override this method if they need to provide
-        more context.
-        """
-        return self.webdriver_class()
-
-
-# FIXME: Missing tests -- vila 2013-04-11
-class RemoteBrowserFactory(BrowserFactory):
-
-    webdriver_class = webdriver.Remote
-
-    def __init__(self, capabilities, remote_url):
-        super(RemoteBrowserFactory, self).__init__()
-        self.capabilities = capabilities
-        self.remote_url = remote_url
-
-    def browser(self):
-        return self.webdriver_class(self.capabilities, self.remote_url)
-
-
-# FIXME: Missing tests -- vila 2013-04-11
-class ChromeFactory(BrowserFactory):
-
-    webdriver_class = webdriver.Chrome
-
-
-# FIXME: Missing tests -- vila 2013-04-11
-class IeFactory(BrowserFactory):
-
-    webdriver_class = webdriver.Ie
-
-
-# FIXME: Missing tests -- vila 2013-04-11
-class PhantomJSFactory(BrowserFactory):
-
-    webdriver_class = webdriver.PhantomJS
-
-
-# FIXME: Missing tests -- vila 2013-04-11
-class OperaFactory(BrowserFactory):
-
-    webdriver_class = webdriver.Opera
-
-
-class FirefoxFactory(BrowserFactory):
-
-    webdriver_class = webdriver.Firefox
-
-    def setup_for_test(self, test):
-        profile = webdriver.FirefoxProfile()
-        profile.set_preference('intl.accept_languages', 'en')
-        if test.assume_trusted_cert_issuer:
-            profile.set_preference('webdriver_assume_untrusted_issuer', False)
-            profile.set_preference(
-                'capability.policy.default.Window.QueryInterface', 'allAccess')
-            profile.set_preference(
-                'capability.policy.default.Window.frameElement.get',
-                'allAccess')
-        if test.javascript_disabled or self.javascript_disabled:
-            profile.set_preference('javascript.enabled', False)
-        self.profile = profile
-
-    def browser(self):
-        return self.webdriver_class(self.profile)
-
-
-# FIXME: Missing tests -- vila 2013-04-11
-browser_factories = {
-    'Chrome': ChromeFactory,
-    'Firefox': FirefoxFactory,
-    'Ie': IeFactory,
-    'Opera': OperaFactory,
-    'PhantomJS': PhantomJSFactory,
-}
-
-
-class SSTTestCase(testtools.TestCase):
-    """A test case that can use the sst framework."""
-
-    xvfb = None
-    xserver_headless = False
-
-    browser_factory = FirefoxFactory()
-
-    javascript_disabled = False
-    assume_trusted_cert_issuer = False
-
-    wait_timeout = 10
-    wait_poll = 0.1
-    base_url = None
-
-    results_directory = _get_full_path('results')
-    screenshots_on = False
-    debug_post_mortem = False
-    extended_report = False
-
-    def shortDescription(self):
-        return None
-
-    def setUp(self):
-        super(SSTTestCase, self).setUp()
-        if self.base_url is not None:
-            actions.set_base_url(self.base_url)
-        actions._set_wait_timeout(self.wait_timeout, self.wait_poll)
-        # Ensures sst.actions will find me
-        actions._test = self
-        if self.xserver_headless and self.xvfb is None:
-            # If we need to run headless and no xvfb is already running, start
-            # a new one for the current test, scheduling the shutdown for the
-            # end of the test.
-            self.xvfb = use_xvfb_server(self)
-        config.results_directory = self.results_directory
-        _make_results_dir()
-        self.start_browser()
-        self.addCleanup(self.stop_browser)
-        if self.screenshots_on:
-            self.addOnException(self.take_screenshot_and_page_dump)
-        if self.debug_post_mortem:
-            self.addOnException(
-                self.print_exception_and_enter_post_mortem)
-        if self.extended_report:
-            self.addOnException(self.report_extensively)
-
-    def start_browser(self):
-        logger.debug('Starting browser')
-        self.browser_factory.setup_for_test(self)
-        self.browser = self.browser_factory.browser()
-        logger.debug('Browser started: %s' % (self.browser.name))
-
-    def stop_browser(self):
-        logger.debug('Stopping browser')
-        self.browser.quit()
-
-    def take_screenshot_and_page_dump(self, exc_info):
-        try:
-            filename = 'screenshot-{0}.png'.format(self.id())
-            actions.take_screenshot(filename)
-        except Exception:
-            # FIXME: Needs to be reported somehow ? -- vila 2012-10-16
-            pass
-        try:
-            # also dump page source
-            filename = 'pagesource-{0}.html'.format(self.id())
-            actions.save_page_source(filename)
-        except Exception:
-            # FIXME: Needs to be reported somehow ? -- vila 2012-10-16
-            pass
-
-    def print_exception_and_enter_post_mortem(self, exc_info):
-        exc_class, exc, tb = exc_info
-        traceback.print_exception(exc_class, exc, tb)
-        pdb.post_mortem(tb)
-
-    def report_extensively(self, exc_info):
-        exc_class, exc, tb = exc_info
-        original_message = str(exc)
-        try:
-            current_url = actions.get_current_url()
-        except Exception:
-            current_url = 'unavailable'
-        try:
-            page_source = actions.get_page_source()
-        except Exception:
-            page_source = 'unavailable'
-        self.addDetail(
-            'Original exception',
-            testtools.content.text_content('{0} : {1}'.format(
-                exc.__class__.__name__, original_message)))
-        self.addDetail('Current url',
-                       testtools.content.text_content(current_url))
-        self.addDetail('Page source',
-                       testtools.content.text_content(page_source))
-
-
-class SSTScriptTestCase(SSTTestCase):
-    """Test case used internally by sst-run and sst-remote."""
-
-    script_dir = '.'
-    script_name = None
-
-    def __init__(self, test_method, context_row=None):
-        super(SSTScriptTestCase, self).__init__('run_test_script')
-        self.test_method = test_method
-        self.id = lambda: '%s.%s.%s' % (self.__class__.__module__,
-                                        self.__class__.__name__, test_method)
-        if context_row is None:
-            context_row = {}
-        self.context = context_row
-
-    def __str__(self):
-        # Since we use run_test_script to encapsulate the call to the
-        # compiled code, we need to override __str__ to get a proper name
-        # reported.
-        return "%s (%s)" % (self.test_method, self.id())
-
-    def shortDescription(self):
-        # The description should be first line of the test method's docstring.
-        # Since we have no real test method here, we override it to always
-        # return none.
-        return None
-
-    def setUp(self):
-        self.script_path = os.path.join(self.script_dir, self.script_name)
-        sys.path.append(self.script_dir)
-        self.addCleanup(sys.path.remove, self.script_dir)
-        self._compile_script()
-        # The script may override some settings. The default value for
-        # JAVASCRIPT_DISABLED and ASSUME_TRUSTED_CERT_ISSUER are False, so if
-        # the user mentions them in his script, it's to turn them on. Also,
-        # getting our hands on the values used in the script is too hackish ;)
-        if 'JAVASCRIPT_DISABLED' in self.code.co_names:
-            self.javascript_disabled = True
-        if 'ASSUME_TRUSTED_CERT_ISSUER' in self.code.co_names:
-            self.assume_trusted_cert_issuer = True
-        super(SSTScriptTestCase, self).setUp()
-        # Start with default values
-        actions.reset_base_url()
-        actions._set_wait_timeout(10, 0.1)
-        # Possibly inject parametrization from associated .csv file
-        previous_context = context.store_context()
-        self.addCleanup(context.restore_context, previous_context)
-        context.populate_context(self.context, self.script_path,
-                                 self.browser.name, self.javascript_disabled)
-
-    def _compile_script(self):
-        with open(self.script_path) as f:
-            source = f.read() + '\n'
-        self.code = compile(source, self.script_path, 'exec')
-
-    def run_test_script(self, result=None):
-        # Run the test catching exceptions sstnam style
-        try:
-            exec self.code in self.context
-        except EndTest:
-            pass
-
-
-def _has_classes(test_dir, entry):
-    """Scan Python source file and check for a class definition."""
-    with open(os.path.join(test_dir, entry)) as f:
-        source = f.read() + '\n'
-    found_classes = []
-
-    def visit_class_def(node):
-        found_classes.append(True)
-
-    node_visitor = ast.NodeVisitor()
-    node_visitor.visit_ClassDef = visit_class_def
-    node_visitor.visit(ast.parse(source))
-    return bool(found_classes)
-
-
-def get_case(test_dir, entry, browser_factory, screenshots_on,
-             context=None, failfast=False, debug=False, extended=False):
-    # our naming convention for tests requires that script-based tests must
-    # not begin with "test_*."  SSTTestCase class-based or other
-    # unittest.TestCase based source files must begin with "test_*".
-    # we also scan the source file to see if it has class definitions,
-    # since script base cases normally don't, but TestCase class-based
-    # tests always will.
-    if entry.startswith('test_') and _has_classes(test_dir, entry):
-        # load just the individual file's tests
-        this_test = defaultTestLoader.discover(test_dir, pattern=entry)
-    else:  # this is for script-based test
-        name = entry[:-3]
-        test_name = 'test_%s' % name
-        this_test = SSTScriptTestCase(test_name, context)
-        this_test.script_dir = test_dir
-        this_test.script_name = entry
-        this_test.browser_factory = browser_factory
-
-        this_test.screenshots_on = screenshots_on
-        this_test.debug_post_mortem = debug
-        this_test.extended_report = extended
-
-    return this_test
-
-
-def get_data(csv_path):
-    """
-    Return a list of data dicts for parameterized testing.
-
-      the first row (headers) match data_map key names.
-      rows beneath are filled with data values.
-    """
-    rows = []
-    print '  Reading data from %r...' % os.path.split(csv_path)[-1],
-    row_num = 0
-    with open(csv_path) as f:
-        headers = f.readline().rstrip().split('^')
-        headers = [header.replace('"', '') for header in headers]
-        headers = [header.replace("'", '') for header in headers]
-        for line in f:
-            row = {}
-            row_num += 1
-            row['_row_num'] = row_num
-            fields = line.rstrip().split('^')
-            for header, field in zip(headers, fields):
-                try:
-                    value = ast.literal_eval(field)
-                except ValueError:
-                    value = field
-                    if value.lower() == 'false':
-                        value = False
-                    if value.lower() == 'true':
-                        value = True
-                row[header] = value
-            rows.append(row)
-    print 'found %s rows' % len(rows)
-    return rows
+                relpath = os.path.dirname(relpath)
+
+    return os.path.abspath(shared_directory)

=== modified file 'src/sst/scripts/remote.py'
--- src/sst/scripts/remote.py	2013-04-18 18:37:10 +0000
+++ src/sst/scripts/remote.py	2013-05-14 11:43:29 +0000
@@ -55,6 +55,9 @@
         failfast=cmd_opts.failfast,
         debug=cmd_opts.debug,
         extended=cmd_opts.extended_tracebacks,
+        # FIXME: not tested -- vila 2013-05-07
+        includes=cmd_opts.includes,
+        excludes=cmd_opts.excludes
     )
 
     print '--------------------------------------------------------------'

=== modified file 'src/sst/scripts/run.py'
--- src/sst/scripts/run.py	2013-04-23 11:01:12 +0000
+++ src/sst/scripts/run.py	2013-05-14 11:43:29 +0000
@@ -33,6 +33,7 @@
 
 import sst
 from sst import (
+    browsers,
     command,
     runtests,
     tests,
@@ -81,7 +82,7 @@
 
     try:
         command.clear_old_results()
-        factory = runtests.browser_factories.get(cmd_opts.browser_type)
+        factory = browsers.browser_factories.get(cmd_opts.browser_type)
         runtests.runtests(
             args,
             test_dir=cmd_opts.dir_name,
@@ -93,6 +94,8 @@
             failfast=cmd_opts.failfast,
             debug=cmd_opts.debug,
             extended=cmd_opts.extended_tracebacks,
+            includes=cmd_opts.includes,
+            excludes=cmd_opts.excludes
         )
     finally:
 
@@ -111,7 +114,9 @@
 
 def run_django(port):
     """Start django server for running local self-tests."""
-    manage_file = './src/testproject/manage.py'
+    here = os.path.abspath(os.path.dirname(__file__))
+    manage_file = os.path.abspath(
+        os.path.join(here, '../../testproject/manage.py'))
     url = 'http://localhost:%s/' % port
 
     if not os.path.isfile(manage_file):
@@ -124,6 +129,9 @@
     if django is None:
         print 'Error: can not find django module.'
         print 'you must have django installed to run the test project.'
+        # FIXME: Using sys.exit() makes it hard to test in isolation. Moreover
+        # this error path is not covered by a test. Both points may be related
+        # ;) -- vila 2013-05-10
         sys.exit(1)
     proc = subprocess.Popen([manage_file, 'runserver', port],
                             stderr=open(os.devnull, 'w'),

=== modified file 'src/sst/selftests/context.py'
--- src/sst/selftests/context.py	2011-08-02 13:11:07 +0000
+++ src/sst/selftests/context.py	2013-05-14 11:43:29 +0000
@@ -8,5 +8,6 @@
 assert __name__ == 'context'
 assert __file__.endswith('context.py')
 
-thisdir = os.path.dirname(__file__)
-assert config.shared_directory == os.path.join(thisdir, 'shared')
+this_dir = os.path.dirname(__file__)
+this_shared = os.path.abspath(os.path.join(this_dir, 'shared'))
+assert (config.shared_directory == this_shared)

=== removed file 'src/sst/selftests/importing.py'
--- src/sst/selftests/importing.py	2012-12-16 15:25:23 +0000
+++ src/sst/selftests/importing.py	1970-01-01 00:00:00 +0000
@@ -1,4 +0,0 @@
-from _module import foo
-
-# the current test directory should be added to sys.path
-assert foo == 3

=== added directory 'src/sst/selftests/regular'
=== added file 'src/sst/selftests/regular/__init__.py'
--- src/sst/selftests/regular/__init__.py	1970-01-01 00:00:00 +0000
+++ src/sst/selftests/regular/__init__.py	2013-05-14 11:43:29 +0000
@@ -0,0 +1,4 @@
+from sst import loader
+
+
+discover = loader.discoverRegularTests

=== renamed file 'src/sst/selftests/test_testtools_testcase.py' => 'src/sst/selftests/regular/test_testtools_testcase.py'
--- src/sst/selftests/test_testtools_testcase.py	2013-02-07 16:01:21 +0000
+++ src/sst/selftests/regular/test_testtools_testcase.py	2013-05-14 11:43:29 +0000
@@ -5,8 +5,5 @@
 
 class TestTestToolsTestCase(TestCase):
 
-    def shortDescription(self):
-        return None
-
     def test_true(self):
         self.assertTrue(True)

=== renamed file 'src/sst/selftests/test_two_methods.py' => 'src/sst/selftests/regular/test_two_methods.py'
--- src/sst/selftests/test_two_methods.py	2013-04-11 14:34:47 +0000
+++ src/sst/selftests/regular/test_two_methods.py	2013-05-14 11:43:29 +0000
@@ -1,7 +1,7 @@
-from sst import runtests
-
-
-class TestBoth(runtests.SSTTestCase):
+from sst import cases
+
+
+class TestBoth(cases.SSTTestCase):
 
     def test_one(self):
         assert True

=== renamed file 'src/sst/selftests/test_unittest_testcase.py' => 'src/sst/selftests/regular/test_unittest_testcase.py'
=== modified file 'src/sst/selftests/shared/helpers.py'
--- src/sst/selftests/shared/helpers.py	2013-04-23 08:42:16 +0000
+++ src/sst/selftests/shared/helpers.py	2013-05-14 11:43:29 +0000
@@ -19,7 +19,9 @@
     # this will copy testproj.db from testproj.db.orginal as setup
     # and remove database after test as a cleanup.
     # (this is used for all tests that access local django admin app only)
-    test_db = 'src/testproject/testproj.db'
+    here = os.path.abspath(os.path.dirname(__file__))
+    test_db = os.path.abspath(
+        os.path.join(here, '..', '..', '..', 'testproject/testproj.db'))
     if os.path.isfile(test_db):
         os.remove(test_db)
     shutil.copyfile(test_db + '.original', test_db)

=== modified file 'src/sst/selftests/static_file.py'
--- src/sst/selftests/static_file.py	2013-04-19 08:03:08 +0000
+++ src/sst/selftests/static_file.py	2013-05-14 11:43:29 +0000
@@ -6,8 +6,8 @@
 import sst.actions
 
 
-static_file = os.path.join(''.join(os.path.split(__file__)[:-1]),
-                           'static.html')
+static_file = os.path.abspath(
+    os.path.join(os.path.dirname(__file__), 'static.html'))
 
 # using full path
 sst.actions.go_to('file:////%s' % static_file)

=== modified file 'src/sst/tests/__init__.py'
--- src/sst/tests/__init__.py	2013-04-23 00:22:48 +0000
+++ src/sst/tests/__init__.py	2013-05-14 11:43:29 +0000
@@ -19,12 +19,14 @@
 import os
 import shutil
 import socket
+import sys
 import tempfile
 
-from sst import runtests
-
-
-class SSTBrowserLessTestCase(runtests.SSTTestCase):
+import testtools
+from sst import cases
+
+
+class SSTBrowserLessTestCase(cases.SSTTestCase):
     """A specialized test class for tests that don't need a browser."""
 
     # We don't use a browser here so disable its use to speed the tests
@@ -36,6 +38,19 @@
         pass
 
 
+class ImportingLocalFilesTest(testtools.TestCase):
+    """Class for tests requiring import of locally generated files.
+
+    This setup the tests working dir in a newly created temp dir and restore
+    sys.modules and sys.path at the end of the test.
+    """
+    def setUp(self):
+        super(ImportingLocalFilesTest, self).setUp()
+        set_cwd_to_tmp(self)
+        protect_imports(self)
+        sys.path.insert(0, self.test_base_dir)
+
+
 def set_cwd_to_tmp(test):
     """Create a temp dir an cd into it for the test duration.
 
@@ -48,6 +63,27 @@
     os.chdir(test.test_base_dir)
 
 
+def protect_imports(test):
+    """Protect sys.modules and sys.path for the test duration.
+
+    This is useful to test imports which modifies sys.modules or requires
+    modifying sys.path.
+    """
+    # Protect sys.modules and sys.path to be able to test imports
+    test.patch(sys, 'path', list(sys.path))
+    orig_modules = sys.modules.copy()
+
+    def cleanup_modules():
+        # Remove all added modules
+        added = [m for m in sys.modules.keys() if m not in orig_modules]
+        if added:
+            for m in added:
+                del sys.modules[m]
+        # Restore deleted or modified modules
+        sys.modules.update(orig_modules)
+    test.addCleanup(cleanup_modules)
+
+
 def check_devserver_port_used(port):
     """check if port is ok to use for django devserver"""
     sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
@@ -61,3 +97,51 @@
     finally:
         sock.close()
     return used
+
+
+def write_tree_from_desc(description):
+    """Write a tree described in a textual form to disk.
+
+    The textual form describes the file contents separated by file/dir names.
+
+    'file: <file name>' on a single line starts a file description. The file
+    name must be the relative path from the tree root.
+
+    'dir: <dir name>' on a single line starts a dir description.
+
+    'link: <link source> <link name>' on a single line describes a symlink to
+    <link source> named <link name>. The source may not exist, spaces are not
+    allowed.
+
+    :param description: A text where files and directories contents is
+        described in a textual form separated by file/dir names.
+    """
+    cur_file = None
+    for line in description.splitlines():
+        if line.startswith('file: '):
+            # A new file begins
+            if cur_file:
+                cur_file.close()
+            cur_file = open(line[len('file: '):], 'w')
+            continue
+        if line.startswith('dir:'):
+            # A new dir begins
+            if cur_file:
+                cur_file.close()
+                cur_file = None
+            os.mkdir(line[len('dir: '):])
+            continue
+        if line.startswith('link: '):
+            # We don't support spaces in names
+            link_desc = line[len('link: '):]
+            try:
+                source, link = link_desc.split()
+            except ValueError:
+                raise ValueError('Invalid link description: %s' % (link_desc,))
+            os.symlink(source, link)
+            continue
+        if cur_file is not None:  # If no file is declared, nothing is written
+            # splitlines() removed the \n, let's add it again
+            cur_file.write(line + '\n')
+    if cur_file:
+        cur_file.close()

=== added file 'src/sst/tests/test_command.py'
--- src/sst/tests/test_command.py	1970-01-01 00:00:00 +0000
+++ src/sst/tests/test_command.py	2013-05-14 11:43:29 +0000
@@ -0,0 +1,62 @@
+#
+#   Copyright (c) 2013 Canonical Ltd.
+#
+#   This file is part of: SST (selenium-simple-test)
+#   https://launchpad.net/selenium-simple-test
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+
+import testtools
+
+from sst import command
+
+
+class TestArgParsing(testtools.TestCase):
+
+    def parse_args(self, provided_args):
+        # command.get_opts_run and friends relies on optparse defaulting to
+        # sys.argv[1:]. To comply with that, we add a dummy first arg to
+        # represent the script name and remove it from the returned args.
+        opts, remaining_args = command.get_opts_run(
+            ['dummy-for-tests'] + provided_args)
+        self.assertEqual('dummy-for-tests', remaining_args[0])
+        return opts, remaining_args[1:]
+
+    def test_default_values(self):
+        opts, args = self.parse_args([])
+        self.assertIs(None, opts.includes)
+        self.assertIs(None, opts.excludes)
+        self.assertEqual([], args)
+
+    def test_single_include(self):
+        opts, args = self.parse_args(['--include', 'foo'])
+        self.assertEquals(['foo'], opts.includes)
+        self.assertIs(None, opts.excludes)
+
+    def test_multiple_includes(self):
+        opts, args = self.parse_args(
+            ['--include', 'foo', '-i', 'bar', '-ibaz'])
+        self.assertEquals(['foo', 'bar', 'baz'], opts.includes)
+        self.assertIs(None, opts.excludes)
+
+    def test_single_exclude(self):
+        opts, args = self.parse_args(['--exclude', 'foo'])
+        self.assertEquals(['foo'], opts.excludes)
+        self.assertIs(None, opts.includes)
+
+    def test_multiple_excludes(self):
+        opts, args = self.parse_args(
+            ['--exclude', 'foo', '-e', 'bar', '-ebaz'])
+        self.assertEquals(['foo', 'bar', 'baz'], opts.excludes)
+        self.assertIs(None, opts.includes)

=== modified file 'src/sst/tests/test_django_devserver.py'
--- src/sst/tests/test_django_devserver.py	2013-04-23 08:42:16 +0000
+++ src/sst/tests/test_django_devserver.py	2013-05-14 11:43:29 +0000
@@ -23,9 +23,10 @@
 
 import testtools
 
+
+import sst
+from sst import tests
 from sst.scripts import run
-from sst.tests import check_devserver_port_used
-from sst import DEVSERVER_PORT
 
 
 class TestDjangoDevServer(testtools.TestCase):
@@ -35,18 +36,19 @@
         # capture test output so we don't pollute the test runs
         self.out = StringIO()
         self.patch(sys, 'stdout', self.out)
+        tests.set_cwd_to_tmp(self)
 
     def test_django_start(self):
-        self.addCleanup(run.kill_django, DEVSERVER_PORT)
-        proc = run.run_django(DEVSERVER_PORT)
+        self.addCleanup(run.kill_django, sst.DEVSERVER_PORT)
+        proc = run.run_django(sst.DEVSERVER_PORT)
         self.assertIsNotNone(proc)
 
     def test_django_devserver_port_used(self):
-        used = check_devserver_port_used(DEVSERVER_PORT)
+        used = tests.check_devserver_port_used(sst.DEVSERVER_PORT)
         self.assertFalse(used)
 
-        self.addCleanup(run.kill_django, DEVSERVER_PORT)
-        run.run_django(DEVSERVER_PORT)
+        self.addCleanup(run.kill_django, sst.DEVSERVER_PORT)
+        run.run_django(sst.DEVSERVER_PORT)
 
-        used = check_devserver_port_used(DEVSERVER_PORT)
+        used = tests.check_devserver_port_used(sst.DEVSERVER_PORT)
         self.assertTrue(used)

=== added file 'src/sst/tests/test_filters.py'
--- src/sst/tests/test_filters.py	1970-01-01 00:00:00 +0000
+++ src/sst/tests/test_filters.py	2013-05-14 11:43:29 +0000
@@ -0,0 +1,142 @@
+#
+#   Copyright (c) 2013 Canonical Ltd.
+#
+#   This file is part of: SST (selenium-simple-test)
+#   https://launchpad.net/selenium-simple-test
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+import re
+import unittest
+
+import testtools
+
+from sst import filters
+
+
+def create_tests_from_ids(ids):
+    suite = unittest.TestSuite()
+
+    def test_id(name):
+        return lambda: name
+
+    for tid in ids:
+        # We need an existing method to create a test. Arbitrarily, we use
+        # id(), that souldn't fail ;) We won't run the test anyway.
+        test = unittest.TestCase(methodName='id')
+        # We can't define the lambda here or 'name' stay bound to the
+        # variable instead of the value, use a proxy to capture the value.
+        test.id = test_id(tid)
+        suite.addTest(test)
+    return suite
+
+
+class TestFilterTestsById(testtools.TestCase):
+
+    def assertFiltered(self, expected, condition, ids):
+        """Check that ``condition`` filters tests created from ``ids``."""
+        filtered = filters.filter_suite(condition, create_tests_from_ids(ids))
+        self.assertEqual(expected,
+                         [t.id() for t in testtools.iterate_tests(filtered)])
+
+    def test_filter_none(self):
+        test_names = ['foo', 'bar']
+        self.assertFiltered(test_names, lambda t: True, test_names)
+
+    def test_filter_all(self):
+        test_names = ['foo', 'bar']
+        self.assertFiltered([], lambda t: False, test_names)
+
+    def test_filter_start(self):
+        self.assertFiltered(['foo', 'footix'],
+                            lambda t: t.id().startswith('foo'),
+                            ['foo', 'footix', 'bar', 'baz', 'fo'])
+
+    def test_filter_in(self):
+        self.assertFiltered(['bar', 'baz'],
+                            lambda t: t.id() in ('bar', 'baz'),
+                            ['foo', 'footix', 'bar', 'baz', 'fo'])
+
+    def test_filter_single(self):
+        self.assertFiltered(['bar'],
+                            lambda t: t.id() == 'bar',
+                            ['foo', 'bar', 'baz'])
+
+    def test_filter_regexp(self):
+        ba = re.compile('ba')
+        self.assertFiltered(['bar', 'baz', 'foobar'],
+                            lambda t: bool(ba.search(t.id())),
+                            ['foo', 'bar', 'baz', 'foobar', 'qux'])
+
+
+class TestFilterTestsByPatterns(testtools.TestCase):
+
+    def assertFiltered(self, expected, patterns, ids):
+        """Check that ``patterns`` filters tests created from ``ids``."""
+        filtered = filters.filter_by_patterns(patterns,
+                                              create_tests_from_ids(ids))
+        self.assertEqual(expected,
+                         [t.id() for t in testtools.iterate_tests(filtered)])
+
+    def test_filter_none(self):
+        self.assertFiltered(['foo', 'bar'], [], ['foo', 'bar'])
+
+    def test_filter_one_pattern(self):
+        self.assertFiltered(['foo', 'foobar', 'barfoo'], ['*foo*'],
+                            ['foo', 'foobar', 'barfoo', 'baz'])
+
+    def test_filter_several_patterns(self):
+        self.assertFiltered(['foo', 'foobar', 'barfoo'], ['foo*', '*arf*'],
+                            ['foo', 'foobar', 'barfoo', 'baz'])
+
+
+class TestFilterTestsByIncludedPrefixes(testtools.TestCase):
+
+    def assertFiltered(self, expected, prefixes, ids):
+        """Check that ``prefixes`` filters tests created from ``ids``."""
+        filtered = filters.include_prefixes(prefixes,
+                                            create_tests_from_ids(ids))
+        self.assertEqual(expected,
+                         [t.id() for t in testtools.iterate_tests(filtered)])
+
+    def test_no_includes(self):
+        self.assertFiltered(['foo', 'bar'], [], ['foo', 'bar'])
+
+    def test_one_include(self):
+        self.assertFiltered(['foo.bar', 'foo.baz'], ['foo'],
+                            ['foo.bar', 'bar', 'foo.baz'])
+
+    def test_several_includes(self):
+        self.assertFiltered(['foo.bar', 'foo.baz', 'bar.baz'], ['foo', 'bar.'],
+                            ['foo.bar', 'bar', 'foo.baz', 'bar.baz'])
+
+
+class TestFilterTestsByExcludedPrefixes(testtools.TestCase):
+
+    def assertFiltered(self, expected, prefixes, ids):
+        """Check that ``prefixes`` filters tests created from ``ids``."""
+        filtered = filters.exclude_prefixes(prefixes,
+                                            create_tests_from_ids(ids))
+        self.assertEqual(expected,
+                         [t.id() for t in testtools.iterate_tests(filtered)])
+
+    def test_no_excludes(self):
+        self.assertFiltered(['foo', 'bar'], [], ['foo', 'bar'])
+
+    def test_one_exclude(self):
+        self.assertFiltered(['bar'], ['foo'],
+                            ['foo.bar', 'bar', 'foo.baz'])
+
+    def test_several_excludes(self):
+        self.assertFiltered(['bar'], ['foo', 'bar.'],
+                            ['foo.bar', 'bar', 'foo.baz', 'bar.baz'])

=== added file 'src/sst/tests/test_loader.py'
--- src/sst/tests/test_loader.py	1970-01-01 00:00:00 +0000
+++ src/sst/tests/test_loader.py	2013-05-14 11:43:29 +0000
@@ -0,0 +1,577 @@
+#
+#   Copyright (c) 2013 Canonical Ltd.
+#
+#   This file is part of: SST (selenium-simple-test)
+#   https://launchpad.net/selenium-simple-test
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+"""Test for sst test loader.
+
+Many tests below create a temporary file hierarchy including python code and/or
+sst scripts. Loading tests imply importing python modules in a way that tests
+can observe via sys.modules while preserving isolation.
+
+The isolation is provided via two means:
+
+- the file hierarchies are created in a temporary directory added to sys.path
+  so test can just import from their current directory,
+
+- tests.protect_imports will remove the loaded modules from sys.modules and
+  restore sys.path.
+
+Because the tests themselves share this module name space, care must be taken
+by tests to not use module names already used in the module. Most of tests
+below therefore use 't' as the main directory because:
+- we use python not lisp so using 't' is ok ;)
+- it's short,
+- it's unlikely to be imported by the module.
+
+"""
+import testtools
+
+from sst import (
+    loader,
+    tests,
+)
+
+
+class TestMatchesForRegexp(testtools.TestCase):
+
+    def test_matches(self):
+        matches = loader.matches_for_regexp('foo.*')
+        # All assertions should succeed, if one of them fails, we have a bigger
+        # problem than having one test for each assertion
+        self.assertTrue(matches('foo'))
+        self.assertFalse(matches('bar'))
+        self.assertTrue(matches('foobar'))
+        self.assertFalse(matches('barfoo'))
+
+
+class TestMatchesForGlob(testtools.TestCase):
+
+    def test_matches(self):
+        matches = loader.matches_for_glob('foo*')
+        # All assertions should succeed, if one of them fails, we have a bigger
+        # problem than having one test for each assertion
+        self.assertTrue(matches('foo'))
+        self.assertFalse(matches('fo'))
+        self.assertFalse(matches('bar'))
+        self.assertTrue(matches('foobar'))
+        self.assertFalse(matches('barfoo'))
+
+
+class TestNameMatcher(testtools.TestCase):
+
+    def test_default_includes(self):
+        name_matcher = loader.NameMatcher()
+        self.assertTrue(name_matcher.includes('foo'))
+        self.assertTrue(name_matcher.matches('foo'))
+
+    def test_default_exclude(self):
+        name_matcher = loader.NameMatcher()
+        self.assertFalse(name_matcher.excludes('foo'))
+        self.assertTrue(name_matcher.matches('foo'))
+
+    def test_provided_includes(self):
+        name_matcher = loader.NameMatcher(
+            includes=loader.matches_for_regexp('^.*foo$'))
+        self.assertTrue(name_matcher.includes('foo'))
+        self.assertTrue(name_matcher.includes('barfoo'))
+        self.assertFalse(name_matcher.includes('bar'))
+        self.assertFalse(name_matcher.includes('foobar'))
+
+    def test_provided_excludes(self):
+        name_matcher = loader.NameMatcher(
+            excludes=loader.matches_for_regexp('^bar.*foo$'))
+        self.assertTrue(name_matcher.excludes('barfoo'))
+        self.assertFalse(name_matcher.excludes('foo'))
+
+
+class TestFileLoader(testtools.TestCase):
+
+    def get_test_loader(self):
+        return loader.TestLoader()
+
+    def test_discover_nothing(self):
+        tests.set_cwd_to_tmp(self)
+        with open('foo', 'w') as f:
+            f.write('bar\n')
+        file_loader = loader.FileLoader(self.get_test_loader())
+        suite = file_loader.discover('.', 'foo')
+        self.assertIs(None, suite)
+
+
+class TestModuleLoader(tests.ImportingLocalFilesTest):
+
+    def get_test_loader(self):
+        return loader.TestLoader()
+
+    def test_default_includes(self):
+        mod_loader = loader.ModuleLoader(self.get_test_loader())
+        self.assertTrue(mod_loader.matches('foo.py'))
+        # But we won't try to import a random file
+        self.assertFalse(mod_loader.matches('foopy'))
+
+    def test_discover_empty_file(self):
+        with open('foo.py', 'w') as f:
+            f.write('')
+        mod_loader = loader.ModuleLoader(self.get_test_loader())
+        suite = mod_loader.discover('.', 'foo.py')
+        self.assertEqual(0, suite.countTestCases())
+
+    def test_discover_invalid_file(self):
+        with open('foo.py', 'w') as f:
+            f.write("I'm no python code")
+        mod_loader = loader.ModuleLoader(self.get_test_loader())
+        self.assertRaises(SyntaxError, mod_loader.discover, '.', 'foo.py')
+
+    def test_discover_valid_file(self):
+        with open('foo.py', 'w') as f:
+            f.write('''
+import unittest
+
+class Test(unittest.TestCase):
+
+    def test_it(self):
+        self.assertTrue(True)
+''')
+        mod_loader = loader.ModuleLoader(self.get_test_loader())
+        suite = mod_loader.discover('.', 'foo.py')
+        self.assertEqual(1, suite.countTestCases())
+
+
+class TestDirLoaderDiscoverPath(tests.ImportingLocalFilesTest):
+
+    def get_test_loader(self):
+        test_loader = loader.TestLoader()
+        # We don't use the default PackageLoader for unit testing DirLoader
+        # behavior. But we still leave ModuleLoader for the file loader.
+        test_loader.dirLoaderClass = loader.DirLoader
+        return test_loader
+
+    def test_discover_path_for_file_without_package(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/foo.py
+I'm not even python code
+''')
+        # Since 'foo' can't be imported, discover_path should not be invoked,
+        # ensure we still get some meaningful error message.
+        dir_loader = loader.DirLoader(self.get_test_loader())
+        e = self.assertRaises(ImportError,
+                              dir_loader.discover_path, 't', 'foo.py')
+        self.assertEqual('No module named t.foo', e.message)
+
+    def test_discover_path_for_valid_file(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/__init__.py
+file: t/foo.py
+import unittest
+
+class Test(unittest.TestCase):
+
+    def test_it(self):
+        self.assertTrue(True)
+''')
+        dir_loader = loader.DirLoader(self.get_test_loader())
+        suite = dir_loader.discover_path('t', 'foo.py')
+        self.assertEqual(1, suite.countTestCases())
+
+    def test_discover_path_for_dir(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/__init__.py
+dir: t/dir
+file: t/dir/foo.py
+import unittest
+
+class Test(unittest.TestCase):
+
+    def test_it(self):
+        self.assertTrue(True)
+''')
+        dir_loader = loader.DirLoader(self.get_test_loader())
+        e = self.assertRaises(ImportError,
+                              dir_loader.discover_path, 't', 'dir')
+        # 't' is a module but 'dir' is not, hence, 'dir.foo' is not either,
+        # blame python for the approximate message ;-/
+        self.assertEqual('No module named dir.foo', e.message)
+
+    def test_discover_path_for_not_matching_symlink(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/foo
+tagada
+link: t/foo t/bar.py
+''')
+        dir_loader = loader.DirLoader(self.get_test_loader())
+        suite = dir_loader.discover_path('t', 'bar.py')
+        self.assertIs(None, suite)
+
+    def test_discover_path_for_broken_symlink(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/foo
+tagada
+link: bar t/qux
+''')
+        dir_loader = loader.DirLoader(self.get_test_loader())
+        suite = dir_loader.discover_path('t', 'qux')
+        self.assertIs(None, suite)
+
+    def test_discover_simple_file_in_dir(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/__init__.py
+file: t/foo.py
+import unittest
+
+class Test(unittest.TestCase):
+
+    def test_it(self):
+        self.assertTrue(True)
+''')
+        dir_loader = loader.DirLoader(self.get_test_loader())
+        suite = dir_loader.discover('.', 't')
+        # Despite using DirLoader, python triggers the 't' import so we are
+        # able to import foo.py and all is well
+        self.assertEqual(1, suite.countTestCases())
+
+
+class TestPackageLoader(tests.ImportingLocalFilesTest):
+
+    def get_test_loader(self):
+        test_loader = loader.TestLoader()
+        return test_loader
+
+    def test_discover_package_with_invalid_file(self):
+        tests.write_tree_from_desc('''dir: dir
+file: dir/__init__.py
+file: dir/foo.py
+I'm not even python code
+''')
+        pkg_loader = loader.PackageLoader(self.get_test_loader())
+        e = self.assertRaises(SyntaxError, pkg_loader.discover, '.', 'dir')
+        self.assertEqual('EOL while scanning string literal', e.args[0])
+
+    def test_discover_simple_file_in_dir(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/__init__.py
+file: t/foo.py
+import unittest
+
+class Test(unittest.TestCase):
+
+    def test_it(self):
+        self.assertTrue(True)
+''')
+        dir_loader = loader.DirLoader(self.get_test_loader())
+        suite = dir_loader.discover('.', 't')
+        self.assertEqual(1, suite.countTestCases())
+
+
+class TestLoadScript(testtools.TestCase):
+
+    def setUp(self):
+        super(TestLoadScript, self).setUp()
+        tests.set_cwd_to_tmp(self)
+
+    def create_script(self, path, content):
+        with open(path, 'w') as f:
+            f.write(content)
+
+    def test_load_simple_script(self):
+        # A simple do nothing script with no imports
+        self.create_script('foo.py', 'pass')
+        suite = loader.TestLoader().loadTestsFromScript('.', 'foo.py')
+        self.assertEqual(1, suite.countTestCases())
+
+    def test_load_simple_script_with_csv(self):
+        self.create_script('foo.py', "pass")
+        with open('foo.csv', 'w') as f:
+            f.write('''\
+'foo'^'bar'
+1^baz
+2^qux
+''')
+        suite = loader.TestLoader().loadTestsFromScript('.', 'foo.py')
+        self.assertEqual(2, suite.countTestCases())
+
+    def test_load_non_existing_script(self):
+        suite = loader.TestLoader().loadTestsFromScript('.', 'foo.py')
+        self.assertEqual(0, suite.countTestCases())
+
+
+class TestScriptLoader(tests.ImportingLocalFilesTest):
+
+    def get_test_loader(self):
+        return loader.TestLoader()
+
+    def test_simple_script(self):
+        tests.write_tree_from_desc('''dir: t
+# no t/__init__.py required, we don't need to import the scripts
+file: t/foo.py
+from sst.actions import *
+
+raise AssertionError('Loading only, executing fails')
+''')
+        script_loader = loader.ScriptLoader(self.get_test_loader())
+        suite = script_loader.discover('t', 'foo.py')
+        self.assertEqual(1, suite.countTestCases())
+
+    def test_ignore_privates(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/_private.py
+''')
+        script_loader = loader.ScriptLoader(self.get_test_loader())
+        suite = script_loader.discover('t', '_private.py')
+        self.assertIs(None, suite)
+
+
+class TesScriptDirLoader(tests.ImportingLocalFilesTest):
+
+    def test_shared(self):
+        tests.write_tree_from_desc('''dir: t
+# no t/__init__.py required, we don't need to import the scripts
+file: t/foo.py
+from sst.actions import *
+
+raise AssertionError('Loading only, executing fails')
+dir: t/shared
+file: t/shared/amodule.py
+Don't look at me !
+''')
+        script_dir_loader = loader.ScriptDirLoader(loader.TestLoader())
+        suite = script_dir_loader.discover('t', 'shared')
+        self.assertIs(None, suite)
+
+    def test_regular(self):
+        tests.write_tree_from_desc('''dir: t
+# no t/__init__.py required, we don't need to import the scripts
+dir: t/subdir
+file: t/subdir/foo.py
+raise AssertionError('Loading only, executing fails')
+dir: t/shared
+file: t/shared/amodule.py
+Don't look at me !
+''')
+        test_loader = loader.TestLoader()
+        suite = test_loader.discoverTests(
+            '.', file_loader_class=loader.ScriptLoader,
+            dir_loader_class=loader.ScriptDirLoader)
+        self.assertEqual(1, suite.countTestCases())
+
+
+class TestTestLoader(tests.ImportingLocalFilesTest):
+
+    def test_simple_file_in_a_dir(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/__init__.py
+file: t/foo.py
+import unittest
+
+class Test(unittest.TestCase):
+
+    def test_me(self):
+      self.assertTrue(True)
+''')
+        test_loader = loader.TestLoader()
+        suite = test_loader.discoverTests('t')
+        self.assertEqual(1, suite.countTestCases())
+
+    def test_broken_file_in_a_dir(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/__init__.py
+file: t/foo.py
+I'm not even python code
+''')
+        test_loader = loader.TestLoader()
+        e = self.assertRaises(SyntaxError, test_loader.discoverTests, 't')
+        self.assertEqual('EOL while scanning string literal', e.args[0])
+
+    def test_scripts_below_regular(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/__init__.py
+file: t/foo.py
+import unittest
+
+class Test(unittest.TestCase):
+
+    def test_me(self):
+      self.assertTrue(True)
+dir: t/scripts
+file: t/scripts/__init__.py
+from sst import loader
+
+discover = loader.discoverTestScripts
+file: t/scripts/script.py
+raise AssertionError('Loading only, executing fails')
+''')
+        test_loader = loader.TestLoader()
+        suite = test_loader.discoverTests('t')
+        self.assertEqual(2, suite.countTestCases())
+        # Check which kind of tests have been discovered or we may miss regular
+        # test cases seen as scripts.
+        self.assertEqual(['t.foo.Test.test_me',
+                          't.scripts.script'],
+                         [t.id() for t in testtools.iterate_tests(suite)])
+
+    def test_regular_below_scripts(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/__init__.py
+dir: t/regular
+file: t/regular/__init__.py
+from sst import loader
+import unittest
+
+discover = loader.discoverRegularTests
+
+class Test(unittest.TestCase):
+
+    def test_in_init(self):
+      self.assertTrue(True)
+file: t/regular/foo.py
+import unittest
+
+class Test(unittest.TestCase):
+
+    def test_me(self):
+      self.assertTrue(True)
+file: t/script.py
+raise AssertionError('Loading only, executing fails')
+''')
+        test_loader = loader.TestLoader()
+        suite = test_loader.discoverTests(
+            't',
+            file_loader_class=loader.ScriptLoader,
+            dir_loader_class=loader.ScriptDirLoader)
+        # Check which kind of tests have been discovered or we may miss regular
+        # test cases seen as scripts.
+        self.assertEqual(['t.regular.Test.test_in_init',
+                          't.regular.foo.Test.test_me',
+                          't.script'],
+                         [t.id() for t in testtools.iterate_tests(suite)])
+
+    def test_regular_and_scripts_mixed(self):
+        def regular(dir_name, name, suffix=None):
+            if suffix is None:
+                suffix = ''
+            return '''
+file: {dir_name}/{name}{suffix}
+from sst import cases
+
+class Test_{name}(cases.SSTTestCase):
+    def test_{name}(self):
+        pass
+'''.format(**locals())
+
+        tests.write_tree_from_desc('''dir: tests
+file: tests/__init__.py
+from sst import loader
+
+discover = loader.discoverRegularTests
+''')
+        tests.write_tree_from_desc(regular('tests', 'test_real', '.py'))
+        tests.write_tree_from_desc(regular('tests', 'test_real1', '.py'))
+        tests.write_tree_from_desc(regular('tests', 'test_real2', '.py'))
+        # Leading '_' => ignored
+        tests.write_tree_from_desc(regular('tests', '_hidden', '.py'))
+        # Not a python file => ignored
+        tests.write_tree_from_desc(regular('tests', 'not-python'))
+        # Some empty files
+        tests.write_tree_from_desc('''
+file: script1.py
+file: script2.py
+file: not_a_test
+# '.p' is intentional, not a typoed '.py'
+file: test_not_a_test.p
+_hidden_too.py
+''')
+        test_loader = loader.TestLoader()
+        suite = test_loader.discoverTests(
+            '.',
+            file_loader_class=loader.ScriptLoader,
+            dir_loader_class=loader.ScriptDirLoader)
+        self.assertEqual(['script1',
+                          'script2',
+                          'tests.test_real.Test_test_real.test_test_real',
+                          'tests.test_real1.Test_test_real1.test_test_real1',
+                          'tests.test_real2.Test_test_real2.test_test_real2'],
+                         [t.id() for t in testtools.iterate_tests(suite)])
+
+
+class TestTestLoaderPattern(tests.ImportingLocalFilesTest):
+
+    def test_default_pattern(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/__init__.py
+file: t/foo.py
+Don't look at me !
+file: t/test_foo.py
+import unittest
+
+class Test(unittest.TestCase):
+
+    def test_me(self):
+      self.assertTrue(True)
+''')
+        test_loader = loader.TestLoader()
+        suite = test_loader.discover('t')
+        self.assertEqual(1, suite.countTestCases())
+
+    def test_pattern(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/__init__.py
+file: t/foo_foo.py
+import unittest
+
+class Test(unittest.TestCase):
+
+    def test_me(self):
+      self.assertTrue(True)
+file: t/test_foo.py
+Don't look at me !
+''')
+        test_loader = loader.TestLoader()
+        suite = test_loader.discover('t', pattern='foo*.py')
+        self.assertEqual(1, suite.countTestCases())
+
+
+class TestTestLoaderTopLevelDir(testtools.TestCase):
+
+    def setUp(self):
+        super(TestTestLoaderTopLevelDir, self).setUp()
+        # We build trees rooted in test_base_dir from which we will import
+        tests.set_cwd_to_tmp(self)
+        tests.protect_imports(self)
+
+    def _create_foo_in_tests(self):
+        tests.write_tree_from_desc('''dir: t
+file: t/__init__.py
+file: t/foo.py
+import unittest
+
+class Test(unittest.TestCase):
+
+    def test_me(self):
+      self.assertTrue(True)
+''')
+
+    def test_simple_file_in_a_dir(self):
+        self._create_foo_in_tests()
+        test_loader = loader.TestLoader()
+        suite = test_loader.discover('t', '*.py', self.test_base_dir)
+        self.assertEqual(1, suite.countTestCases())
+
+    def test_simple_file_in_a_dir_no_sys_path(self):
+        self._create_foo_in_tests()
+        test_loader = loader.TestLoader()
+        e = self.assertRaises(ImportError,
+                              test_loader.discover, 't', '*.py')
+        self.assertEqual(e.message, 'No module named t')

=== added file 'src/sst/tests/test_protect_imports.py'
--- src/sst/tests/test_protect_imports.py	1970-01-01 00:00:00 +0000
+++ src/sst/tests/test_protect_imports.py	2013-05-14 11:43:29 +0000
@@ -0,0 +1,86 @@
+#
+#   Copyright (c) 2013 Canonical Ltd.
+#
+#   This file is part of: SST (selenium-simple-test)
+#   https://launchpad.net/selenium-simple-test
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+
+import sys
+import testtools
+
+from sst import tests
+
+
+class TestProtectImports(testtools.TestCase):
+
+    def setUp(self):
+        super(TestProtectImports, self).setUp()
+        tests.protect_imports(self)
+
+    def run_successful_test(self, test):
+        result = testtools.TestResult()
+        test.run(result)
+        self.assertTrue(result.wasSuccessful())
+
+    def test_add_module(self):
+        self.assertIs(None, sys.modules.get('foo', None))
+
+        class Test(testtools.TestCase):
+
+            def test_it(self):
+                tests.protect_imports(self)
+                sys.modules['foo'] = 'bar'
+
+        self.run_successful_test(Test('test_it'))
+        self.assertIs(None, sys.modules.get('foo', None))
+
+    def test_remove_module(self):
+        self.assertIs(None, sys.modules.get('I_dont_exist', None))
+        sys.modules['I_dont_exist'] = 'bar'
+
+        class Test(testtools.TestCase):
+
+            def test_it(self):
+                tests.protect_imports(self)
+                self.assertEqual('bar', sys.modules['I_dont_exist'])
+                del sys.modules['I_dont_exist']
+        self.run_successful_test(Test('test_it'))
+        self.assertEqual('bar', sys.modules['I_dont_exist'])
+
+    def test_modify_module(self):
+        self.assertIs(None, sys.modules.get('I_dont_exist', None))
+        sys.modules['I_dont_exist'] = 'bar'
+
+        class Test(testtools.TestCase):
+
+            def test_it(self):
+                tests.protect_imports(self)
+                self.assertEqual('bar', sys.modules['I_dont_exist'])
+                sys.modules['I_dont_exist'] = 'qux'
+        self.run_successful_test(Test('test_it'))
+        self.assertEqual('bar', sys.modules['I_dont_exist'])
+
+    def test_sys_path_restored(self):
+        tests.set_cwd_to_tmp(self)
+        inserted = self.test_base_dir
+        self.assertFalse(inserted in sys.path)
+
+        class Test(testtools.TestCase):
+
+            def test_it(self):
+                tests.protect_imports(self)
+                sys.path.insert(0, inserted)
+        self.run_successful_test(Test('test_it'))
+        self.assertFalse(inserted in sys.path)

=== renamed file 'src/sst/tests/test_results_output.py' => 'src/sst/tests/test_result.py'
--- src/sst/tests/test_results_output.py	2013-05-07 13:58:29 +0000
+++ src/sst/tests/test_result.py	2013-05-14 11:43:29 +0000
@@ -18,166 +18,268 @@
 #
 
 
-import os
-import unittest
 from cStringIO import StringIO
+import sys
 
 import junitxml
 import testtools
 
 from sst import (
+    result,
     tests,
-    runtests,
 )
 
 
-def _make_pass_test_suite():
-    class InnerPassTestCase(unittest.TestCase):
-        def test_inner_pass(self):
-            self.assertTrue(True)
-    suite = unittest.TestSuite()
-    suite.addTest(InnerPassTestCase('test_inner_pass'))
-    return suite
-
-
-def _make_fail_test_suite():
-    class InnerFailTestCase(unittest.TestCase):
-        def test_inner_fail(self):
+def get_case(kind):
+    # Define the class in a function so test loading don't try to load it as a
+    # regular test class.
+    class Test(testtools.TestCase):
+
+        def test_pass(self):
+            pass
+
+        def test_fail(self):
             self.assertTrue(False)
-    suite = unittest.TestSuite()
-    suite.addTest(InnerFailTestCase('test_inner_fail'))
-    return suite
-
-
-def _make_error_test_suite():
-    class InnerErrorTestCase(unittest.TestCase):
-        def test_inner_error(self):
-            raise
-    suite = unittest.TestSuite()
-    suite.addTest(InnerErrorTestCase('test_inner_error'))
-    return suite
-
-
-def _make_skip_test_suite():
-    class InnerSkipTestCase(unittest.TestCase):
-        @testtools.skip('skip me')
-        def test_inner_skip(self):
-            pass
-    suite = unittest.TestSuite()
-    suite.addTest(InnerSkipTestCase('test_inner_skip'))
-    return suite
-
-
-class ConsoleOutputTestCase(testtools.TestCase):
-
-    def setUp(self):
-        super(ConsoleOutputTestCase, self).setUp()
-        tests.set_cwd_to_tmp(self)
-        self.out = StringIO()
-        self.text_result = runtests.TextTestResult(self.out)
-
-    def assert_output(self, suite, regex):
-        suite.run(self.text_result)
-        self.console_output = self.out.getvalue()
-        self.assertRegexpMatches(self.console_output, regex)
-
-    def test_text_output_pass(self):
-        suite = _make_pass_test_suite()
-        regex = (
-            r'test_inner_pass '
-            r'\(sst.tests.test_results_output.InnerPassTestCase\) ...'
-            r'\nOK \([0-9]*.[0-9]{3} secs\)\n'
-        )
-        self.assert_output(suite, regex)
-
-    def test_text_output_fail(self):
-        suite = _make_fail_test_suite()
-        regex = (
-            r'test_inner_fail '
-            r'\(sst.tests.test_results_output.InnerFailTestCase\) ...'
-            r'\nFAIL\n'
-        )
-        self.assert_output(suite, regex)
-
-    def test_text_output_error(self):
-        suite = _make_error_test_suite()
-        regex = (
-            r'test_inner_error '
-            r'\(sst.tests.test_results_output.InnerErrorTestCase\) ...'
-            r'\nERROR\n'
-        )
-        self.assert_output(suite, regex)
-
-    def test_text_output_skip(self):
-        suite = _make_skip_test_suite()
-        regex = (
-            r'test_inner_skip '
-            r'\(sst.tests.test_results_output.InnerSkipTestCase\) ...'
-            r'\nSkipped \'skip me\'\n'
-        )
-        self.assert_output(suite, regex)
-
-
-class XmlOutputTestCase(testtools.TestCase):
-
-    def setUp(self):
-        super(XmlOutputTestCase, self).setUp()
-        tests.set_cwd_to_tmp(self)
-        self.results_file = 'results.xml'
-        self.xml_stream = file(self.results_file, 'wb')
-        self.xml_result = junitxml.JUnitXmlResult(self.xml_stream)
-        self.addCleanup(os.remove, 'results.xml')
-
-    def assert_output(self, suite, regex):
-        suite.run(self.xml_result)
-        # need this to signal junitxml or no results get written
-        self.xml_result.stopTestRun()
-        self.xml_stream.close()
-        self.assertIn(self.results_file, os.listdir(self.test_base_dir))
-        with open(self.results_file) as f:
-            content = f.read()
-        self.assertRegexpMatches(content, regex)
-
-    def test_xml_output_pass(self):
-        suite = _make_pass_test_suite()
-        regex = (
-            r'<testsuite errors="0" failures="0" name="" tests="1" '
-            r'time=".*">\n'
-            r'<testcase '
-            r'classname="sst.tests.test_results_output.InnerPassTestCase" '
-            r'name="test_inner_pass" time="[0-9]*.[0-9]{3}"'
-        )
-        self.assert_output(suite, regex)
-
-    def test_xml_output_fail(self):
-        suite = _make_fail_test_suite()
-        regex = (
-            r'<testsuite errors="0" failures="1" name="" tests="1" '
-            r'time=".*">\n'
-            r'<testcase '
-            r'classname="sst.tests.test_results_output.InnerFailTestCase" '
-            r'name="test_inner_fail" time="[0-9]*.[0-9]{3}"'
-        )
-        self.assert_output(suite, regex)
-
-    def test_xml_output_error(self):
-        suite = _make_error_test_suite()
-        regex = (
-            r'<testsuite errors="1" failures="0" name="" tests="1" '
-            r'time=".*">\n'
-            r'<testcase '
-            r'classname="sst.tests.test_results_output.InnerErrorTestCase" '
-            r'name="test_inner_error" time="[0-9]*.[0-9]{3}"'
-        )
-        self.assert_output(suite, regex)
-
-    def test_xml_output_skip(self):
-        suite = _make_skip_test_suite()
-        regex = (
-            r'<testsuite errors="0" failures="0" name="" tests="1" '
-            r'time=".*">\n<testcase classname="sst.tests.'
-            r'test_results_output.InnerSkipTestCase" '
-            r'name="test_inner_skip" time="[0-9]*.[0-9]{3}"'
-            r'>\n<skipped>skip me</skipped>\n</testcase>\n</testsuite>'
-        )
-        self.assert_output(suite, regex)
+
+        def test_error(self):
+            raise SyntaxError
+
+        def test_skip(self):
+            self.skipTest('')
+
+        def test_skip_reason(self):
+            self.skipTest('Because')
+
+        def test_expected_failure(self):
+            # We expect the test to fail and it does
+            self.expectFailure("1 should be 0", self.assertEqual, 1, 0)
+
+        def test_unexpected_success(self):
+            # We expect the test to fail but it doesn't
+            self.expectFailure("1 is not 1", self.assertEqual, 1, 1)
+
+    test_method = 'test_%s' % (kind,)
+    return Test(test_method)
+
+
+class TestConsoleOutput(testtools.TestCase):
+
+    def setUp(self):
+        super(TestConsoleOutput, self).setUp()
+        tests.set_cwd_to_tmp(self)
+
+    def assertOutput(self, expected, kind):
+        # We don't care about timing here so we always return 0 which
+        # simplifies matching the expected result
+        test = get_case(kind)
+        out = StringIO()
+        res = result.TextTestResult(out, timer=lambda: 0.0)
+        test.run(res)
+        self.assertEquals(expected, res.stream.getvalue())
+
+    def test_pass_output(self):
+        self.assertOutput('.', 'pass')
+
+    def test_fail_output(self):
+        self.assertOutput('F', 'fail')
+
+    def test_error_output(self):
+        self.assertOutput('E', 'error')
+
+    def test_skip_output(self):
+        self.assertOutput('s', 'skip')
+
+    def test_skip_reason_output(self):
+        self.assertOutput('s', 'skip_reason')
+
+    def test_expected_failure_output(self):
+        self.assertOutput('x', 'expected_failure')
+
+    def test_unexpected_success_output(self):
+        self.assertOutput('u', 'unexpected_success')
+
+
+class TestVerboseConsoleOutput(testtools.TestCase):
+
+    def setUp(self):
+        super(TestVerboseConsoleOutput, self).setUp()
+        tests.set_cwd_to_tmp(self)
+
+    def assertOutput(self, expected, kind):
+        # We don't care about timing here so we always return 0 which
+        # simplifies matching the expected result
+        test = get_case(kind)
+        out = StringIO()
+        res = result.TextTestResult(out, verbosity=2, timer=lambda: 0.0)
+        test.run(res)
+        self.assertEquals(expected, res.stream.getvalue())
+
+    def test_pass_output(self):
+        self.assertOutput('''\
+test_pass (sst.tests.test_result.Test) ... OK (0.000 secs)
+''',
+                          'pass')
+
+    def test_fail_output(self):
+        self.assertOutput('''\
+test_fail (sst.tests.test_result.Test) ... FAIL (0.000 secs)
+''',
+                          'fail')
+
+    def test_error_output(self):
+        self.assertOutput('''\
+test_error (sst.tests.test_result.Test) ... ERROR (0.000 secs)
+''',
+                          'error')
+
+    def test_skip_output(self):
+        self.assertOutput('''\
+test_skip (sst.tests.test_result.Test) ... SKIP (0.000 secs)
+''',
+                          'skip')
+
+    def test_skip_reason_output(self):
+        self.assertOutput('''\
+test_skip_reason (sst.tests.test_result.Test) ... SKIP Because (0.000 secs)
+''',
+                          'skip_reason')
+
+    def test_expected_failure_output(self):
+        self.assertOutput('''\
+test_expected_failure (sst.tests.test_result.Test) ... XFAIL (0.000 secs)
+''',
+                          'expected_failure')
+
+    def test_unexpected_success_output(self):
+        self.assertOutput('''\
+test_unexpected_success (sst.tests.test_result.Test) ... NOTOK (0.000 secs)
+''',
+                          'unexpected_success')
+
+
+class TestXmlOutput(testtools.TestCase):
+
+    def setUp(self):
+        super(TestXmlOutput, self).setUp()
+        tests.set_cwd_to_tmp(self)
+
+    def assertOutput(self, template, kind, kwargs=None):
+        """Assert the expected output from a run for a given test.
+
+        :param template: A string where common strings have been replaced by a
+            keyword so we don't run into pep8 warnings for long lines.
+
+        :param kind: A string used to select the kind of test.
+
+        :param kwargs: A dict with more keywords for the template. This allows
+            some tests to add more keywords when they are test specific.
+        """
+        if kwargs is None:
+            kwargs = dict()
+        out = StringIO()
+        res = junitxml.JUnitXmlResult(out)
+        # We don't care about timing here so we always return 0 which
+        # simplifies matching the expected result
+        res._now = lambda: 0.0
+        res._duration = lambda f: 0.0
+        test = get_case(kind)
+        test.run(res)
+        # due to the nature of JUnit XML output, nothing will be written to
+        # the stream until stopTestRun() is called.
+        res.stopTestRun()
+        # To allow easier reading for template, we format some known values
+        kwargs.update(dict(classname='%s.%s' % (test.__class__.__module__,
+                                                test.__class__.__name__),
+                           name=test._testMethodName))
+        expected = template.format(**kwargs)
+        self.assertEquals(expected, res._stream.getvalue())
+
+    def test_pass_output(self):
+        self.assertOutput('''\
+<testsuite errors="0" failures="0" name="" tests="1" time="0.000">
+<testcase classname="{classname}" name="{name}" time="0.000"/>
+</testsuite>
+''',
+                          'pass')
+
+    def test_fail_output(self):
+        # Getting the file name right is tricky, depending on whether the
+        # module was just recompiled or not __file__ can be either .py or .pyc
+        # but when it appears in an exception, the .py is always used.
+        filename = __file__.replace('.pyc', '.py').replace('.pyo', '.py')
+        more = dict(exc_type='testtools.testresult.real._StringException',
+                    filename=filename)
+        self.assertOutput('''\
+<testsuite errors="0" failures="1" name="" tests="1" time="0.000">
+<testcase classname="{classname}" name="{name}" time="0.000">
+<failure type="{exc_type}">_StringException: Traceback (most recent call last):
+  File "{filename}", line 42, in {name}
+    self.assertTrue(False)
+AssertionError: False is not true
+
+</failure>
+</testcase>
+</testsuite>
+''',
+                          'fail',
+                          more)
+
+    def test_error_output(self):
+        # Getting the file name right is tricky, depending on whether the
+        # module was just recompiled or not __file__ can be either .py or .pyc
+        # but when it appears in an exception, the .py is always used.
+        filename = __file__.replace('.pyc', '.py').replace('.pyo', '.py')
+        more = dict(exc_type='testtools.testresult.real._StringException',
+                    filename=filename)
+        self.assertOutput('''\
+<testsuite errors="1" failures="0" name="" tests="1" time="0.000">
+<testcase classname="{classname}" name="{name}" time="0.000">
+<error type="{exc_type}">_StringException: Traceback (most recent call last):
+  File "{filename}", line 45, in {name}
+    raise SyntaxError
+SyntaxError: None
+
+</error>
+</testcase>
+</testsuite>
+''',
+                          'error',
+                          more)
+
+    def test_skip_output(self):
+        self.assertOutput('''\
+<testsuite errors="0" failures="0" name="" tests="1" time="0.000">
+<testcase classname="{classname}" name="{name}" time="0.000">
+<skipped></skipped>
+</testcase>
+</testsuite>
+''',
+                          'skip')
+
+    def test_skip_reason_output(self):
+        self.assertOutput('''\
+<testsuite errors="0" failures="0" name="" tests="1" time="0.000">
+<testcase classname="{classname}" name="{name}" time="0.000">
+<skipped>Because</skipped>
+</testcase>
+</testsuite>
+''',
+                          'skip_reason')
+
+    def test_expected_failure_output(self):
+        self.assertOutput('''\
+<testsuite errors="0" failures="0" name="" tests="1" time="0.000">
+<testcase classname="{classname}" name="{name}" time="0.000"/>
+</testsuite>
+''',
+                          'expected_failure')
+
+    def test_unexpected_success_output(self):
+        self.assertOutput('''\
+<testsuite errors="0" failures="1" name="" tests="1" time="0.000">
+<testcase classname="{classname}" name="{name}" time="0.000">
+<failure type="unittest.case._UnexpectedSuccess"/>
+</testcase>
+</testsuite>
+''',
+                          'unexpected_success')

=== added file 'src/sst/tests/test_runtests.py'
--- src/sst/tests/test_runtests.py	1970-01-01 00:00:00 +0000
+++ src/sst/tests/test_runtests.py	2013-05-14 11:43:29 +0000
@@ -0,0 +1,92 @@
+#
+#   Copyright (c) 2013 Canonical Ltd.
+#
+#   This file is part of: SST (selenium-simple-test)
+#   https://launchpad.net/selenium-simple-test
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+
+from cStringIO import StringIO
+import sys
+
+from sst import (
+    runtests,
+    tests,
+)
+
+
+class TestRunTestsFilteringByTestId(tests.ImportingLocalFilesTest):
+
+    def setUp(self):
+        super(TestRunTestsFilteringByTestId, self).setUp()
+        tests.write_tree_from_desc('''dir: t
+file: t/__init__.py
+file: t/test_foo.py
+import unittest
+
+class Test(unittest.TestCase):
+
+    def test_me(self):
+      self.assertTrue(True)
+file: t/bar.py
+Don't look at me !
+file: t/too.py
+Don't look at me !
+''')
+
+    def run_tests(self, *args, **kwargs):
+        # FIXME: runtests use print, it should accept a stream instead. We also
+        # should be able to better focus the test filtering but that requires
+        # refactoring runtests. -- vila 2013-05-07
+        self.out = StringIO()
+        self.patch(sys, 'stdout', self.out)
+        runtests.runtests(*args, test_dir='t', collect_only=True, **kwargs)
+        lines = self.out.getvalue().splitlines()
+        self.assertEqual('', lines[0])
+        # We don't care about the number of tests, that will be checked later
+        # by the caller
+        self.assertTrue(lines[1].endswith('test cases loaded'))
+        self.assertEqual('', lines[2])
+        self.assertEqual('-' * 62, lines[3])
+        self.assertEqual('Collect-Only Enabled, Not Running Tests...',
+                         lines[4])
+        self.assertEqual('', lines[5])
+        self.assertEqual('Tests Collected:', lines[6])
+        self.assertEqual('----------------', lines[7])
+        return lines[8:]
+
+    def test_all(self):
+        self.assertEqual(['t.bar', 't.test_foo', 't.too'],
+                         self.run_tests(None))
+
+    def test_single_include(self):
+        self.assertEqual(['t.bar'],
+                         self.run_tests(None, includes=['t.b']))
+
+    def test_multiple_includes(self):
+        self.assertEqual(['t.bar', 't.too'],
+                         self.run_tests(None, includes=['t.b', 't.to']))
+
+    def test_single_exclude(self):
+        self.assertEqual(['t.bar'],
+                         self.run_tests(None, excludes=['t.t']))
+
+    def test_multiple_excludes(self):
+        self.assertEqual(['t.test_foo'],
+                         self.run_tests(None, excludes=['t.to', 't.b']))
+
+    def test_mixed(self):
+        self.assertEqual(['t.test_foo'],
+                         self.run_tests(None, includes=['t.t'],
+                                        excludes=['t.to']))

=== removed file 'src/sst/tests/test_runtests_find_cases.py'
--- src/sst/tests/test_runtests_find_cases.py	2013-04-23 08:42:16 +0000
+++ src/sst/tests/test_runtests_find_cases.py	1970-01-01 00:00:00 +0000
@@ -1,124 +0,0 @@
-#
-#   Copyright (c) 2013 Canonical Ltd.
-#
-#   This file is part of: SST (selenium-simple-test)
-#   https://launchpad.net/selenium-simple-test
-#
-#   Licensed under the Apache License, Version 2.0 (the "License");
-#   you may not use this file except in compliance with the License.
-#   You may obtain a copy of the License at
-#
-#       http://www.apache.org/licenses/LICENSE-2.0
-#
-#   Unless required by applicable law or agreed to in writing, software
-#   distributed under the License is distributed on an "AS IS" BASIS,
-#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-#   See the License for the specific language governing permissions and
-#   limitations under the License.
-#
-
-
-import os
-
-import testtools
-
-from sst import tests
-from sst import runtests
-
-
-def _make_empty_files(dir):
-    file_names = (
-        'test_a_real_test.py',
-        'test_a_real_test2.py',
-        'script.py',
-        'not_a_test',
-        'test_not_a_test.p',
-        '_hidden.py',
-    )
-    os.mkdir(dir)
-    for fn in file_names:
-        with open(os.path.join(dir, fn), 'w'):
-            pass
-
-
-class TestFindCases(testtools.TestCase):
-
-    def setUp(self):
-        super(TestFindCases, self).setUp()
-        tests.set_cwd_to_tmp(self)
-        self.cases_dir = os.path.join(self.test_base_dir, 'cases')
-
-    def test_runtests_find_cases_multi_name(self):
-        _make_empty_files(self.cases_dir)
-        args = (
-            'test_a_real_test',
-            'script',
-        )
-        matches = {
-            'test_a_real_test.py',
-            'script.py',
-        }
-        found = runtests.find_cases(args, self.cases_dir)
-        self.assertSetEqual(matches, found)
-
-    def test_runtests_find_cases_single_name(self):
-        _make_empty_files(self.cases_dir)
-        args = (
-            'test_a_real_test',
-        )
-        matches = {
-            'test_a_real_test.py',
-        }
-        found = runtests.find_cases(args, self.cases_dir)
-        self.assertSetEqual(matches, found)
-
-    def test_runtests_find_cases_glob(self):
-        _make_empty_files(self.cases_dir)
-        matches = {
-            'test_a_real_test.py',
-            'test_a_real_test2.py',
-        }
-
-        args = (
-            'test_a_real_test*',
-        )
-        found = runtests.find_cases(args, self.cases_dir)
-        self.assertSetEqual(matches, found)
-
-        args = (
-            'test_*_test*',
-        )
-        found = runtests.find_cases(args, self.cases_dir)
-        self.assertSetEqual(matches, found)
-
-    def test_runtests_find_cases_glob_singlechar(self):
-        _make_empty_files(self.cases_dir)
-        args = (
-            'test_a_real_test?',
-        )
-        matches = {
-            'test_a_real_test2.py',
-        }
-
-        found = runtests.find_cases(args, self.cases_dir)
-        self.assertSetEqual(matches, found)
-
-    def test_runtests_find_cases_glob_and_name(self):
-        _make_empty_files(self.cases_dir)
-        args = (
-            'test_*',
-            'script',
-        )
-        matches = {
-            'test_a_real_test.py',
-            'test_a_real_test2.py',
-            'script.py',
-        }
-        found = runtests.find_cases(args, self.cases_dir)
-        self.assertSetEqual(matches, found)
-
-    def test_runtests_find_cases_none_found(self):
-        _make_empty_files(self.cases_dir)
-        args = ('xNOMATCHx',)
-        found = runtests.find_cases(args, self.cases_dir)
-        self.assertEqual(len(found), 0)

=== removed file 'src/sst/tests/test_runtests_get_suites.py'
--- src/sst/tests/test_runtests_get_suites.py	2013-04-18 18:29:29 +0000
+++ src/sst/tests/test_runtests_get_suites.py	1970-01-01 00:00:00 +0000
@@ -1,116 +0,0 @@
-#
-#   Copyright (c) 2013 Canonical Ltd.
-#
-#   This file is part of: SST (selenium-simple-test)
-#   https://launchpad.net/selenium-simple-test
-#
-#   Licensed under the Apache License, Version 2.0 (the "License");
-#   you may not use this file except in compliance with the License.
-#   You may obtain a copy of the License at
-#
-#       http://www.apache.org/licenses/LICENSE-2.0
-#
-#   Unless required by applicable law or agreed to in writing, software
-#   distributed under the License is distributed on an "AS IS" BASIS,
-#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-#   See the License for the specific language governing permissions and
-#   limitations under the License.
-#
-
-
-import os
-
-import testtools
-import unittest
-
-from sst import tests
-from sst import runtests
-
-
-def _make_test_files(dir):
-
-    os.mkdir(dir)
-
-    # generate files with TestCase classes
-    testclass_file_names = (
-        'test_a_real_test.py',
-        'test_a_real_test1.py',
-        'test_a_real_test2.py',
-        '_hidden_class.py',
-        'class_hide_case',
-        'class_hiding_case.py',
-    )
-    for fn in testclass_file_names:
-        with open(os.path.join(dir, fn), 'w') as f:
-            f.write('from sst import runtests\n')
-            f.write('class Test_%s(runtests.SSTTestCase):\n' % fn[:-3])
-            f.write('    def test_%s(self): pass\n' % fn[:-3])
-
-    # generate empty files
-    file_names = (
-        '__init__.py',
-        'script1.py',
-        'script2.py',
-        'not_a_test',
-        'test_not_a_test.p',
-        '_hidden2.py'
-    )
-    for fn in file_names:
-        with open(os.path.join(dir, fn), 'w') as f:
-            pass
-
-
-class TestGetSuites(testtools.TestCase):
-
-    def setUp(self):
-        super(TestGetSuites, self).setUp()
-        tests.set_cwd_to_tmp(self)
-        self.cases_dir = os.path.join(self.test_base_dir, 'cases')
-
-    def test_runtests_get_suites(self):
-        _make_test_files(self.cases_dir)
-
-        test_names = ('*', )
-        test_dir = self.cases_dir
-        shared_dir = '.'
-        collect_only = False
-        screenshots_on = False
-        failfast = False,
-        debug = False
-        extended = False
-
-        found = runtests.get_suites(
-            test_names, test_dir, shared_dir, collect_only,
-            runtests.FirefoxFactory(),
-            screenshots_on, failfast, debug, extended
-        )
-        suite = found[0]._tests
-
-        # assert we loaded correct number of cases
-        self.assertEquals(len(suite), 6)
-
-        expected_scripted_tests = (
-            'test_script1',
-            'test_script2',
-            'test_class_hiding_case',
-        )
-        expected_testcase_tests = (
-            'test_test_a_real_test',
-            'test_test_a_real_test1',
-            'test_test_a_real_test2',
-        )
-
-        for test in suite:
-            if issubclass(test.__class__, runtests.SSTTestCase):
-                self.assertIsInstance(test, runtests.SSTTestCase)
-                name = test.id().split('.')[-1]
-                self.assertIn(name, expected_scripted_tests)
-            elif issubclass(test.__class__, unittest.suite.TestSuite):
-                self.assertIsInstance(test, unittest.suite.TestSuite)
-                for test_class in test._tests:
-                    for case in test_class._tests:
-                        for t in case._tests:
-                            name = t.id().split('.')[-1]
-                            self.assertIn(name, expected_testcase_tests)
-            else:
-                raise Exception('Can not identify test')

=== modified file 'src/sst/tests/test_sst_run.py'
--- src/sst/tests/test_sst_run.py	2013-04-23 11:05:53 +0000
+++ src/sst/tests/test_sst_run.py	2013-05-14 11:43:29 +0000
@@ -24,8 +24,8 @@
 import testtools
 
 from sst import (
+    cases,
     config,
-    runtests,
     tests,
 )
 
@@ -34,14 +34,14 @@
 
     def setUp(self):
         super(TestSSTScriptTestCase, self).setUp()
-        self.test = runtests.SSTScriptTestCase('foo')
+        self.test = cases.SSTScriptTestCase('dir', 'foo.py')
 
     def test_id(self):
         """The test id mentions the python class path and the test name."""
         # FIXME: This is a minimal test to cover http://pad.lv/1087606, it
         # would be better to check a results.xml file but we don't have the
         # test infrastructure for that (yet) -- vila 2012-12-07
-        self.assertEqual('sst.runtests.SSTScriptTestCase.foo', self.test.id())
+        self.assertEqual('dir.foo', self.test.id())
 
 
 class TestSSTTestCase(testtools.TestCase):
@@ -68,7 +68,6 @@
         self.assertFalse(test.screenshots_on)
         self.assertEqual(test.wait_poll, 0.1)
         self.assertEqual(test.wait_timeout, 10)
-        self.assertIsNone(test.shortDescription())
         self.assertEqual(test.id(), 'sst.tests.SSTBrowserLessTestCase.run')
 
     def test_config(self):
@@ -77,5 +76,4 @@
         self.assertEqual(config.cache, {})
         self.assertEqual(config.flags, [])
         self.assertFalse(config.javascript_disabled)
-        self.assertEqual(os.path.split(config.results_directory)[-1],
-                         'results')
+        self.assertEqual(os.path.basename(config.results_directory), 'results')

=== modified file 'src/sst/tests/test_sst_script_test_case.py'
--- src/sst/tests/test_sst_script_test_case.py	2013-04-11 14:59:48 +0000
+++ src/sst/tests/test_sst_script_test_case.py	2013-05-14 11:43:29 +0000
@@ -27,17 +27,19 @@
 from testtools import matchers
 
 from sst import (
-    runtests,
+    cases,
     tests,
 )
 
 
-class SSTStringTestCase(runtests.SSTScriptTestCase):
+class SSTStringTestCase(cases.SSTScriptTestCase):
 
-    script_name = 'test_foo'
-    script_code = 'pass'
     xserver_headless = True
 
+    def __init__(self, code='pass'):
+        super(SSTStringTestCase, self).__init__('ignored_dir', 'ignored_name')
+        self.script_code = code
+
     def setUp(self):
         # We don't need to compile the script because we have already define
         # the code to execute.

=== added file 'src/sst/tests/test_write_tree.py'
--- src/sst/tests/test_write_tree.py	1970-01-01 00:00:00 +0000
+++ src/sst/tests/test_write_tree.py	2013-05-14 11:43:29 +0000
@@ -0,0 +1,94 @@
+#
+#   Copyright (c) 2013 Canonical Ltd.
+#
+#   This file is part of: SST (selenium-simple-test)
+#   https://launchpad.net/selenium-simple-test
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+
+import os
+import testtools
+
+from sst import tests
+
+
+class TestWriteTree(testtools.TestCase):
+
+    def setUp(self):
+        super(TestWriteTree, self).setUp()
+        tests.set_cwd_to_tmp(self)
+
+    def test_empty_description(self):
+        self.assertEqual([], os.listdir('.'))
+        tests.write_tree_from_desc('')
+        self.assertEqual([], os.listdir('.'))
+
+    def test_single_line_without_return(self):
+        self.assertEqual([], os.listdir('.'))
+        tests.write_tree_from_desc('file: foo')
+        self.assertEqual(['foo'], os.listdir('.'))
+        self.assertEqual('', file('foo').read())
+
+    def test_leading_line_is_ignored(self):
+        self.assertEqual([], os.listdir('.'))
+        tests.write_tree_from_desc('tagada\nfile: foo')
+        self.assertEqual(['foo'], os.listdir('.'))
+        self.assertEqual('', file('foo').read())
+
+    def test_orphan_line_is_ignored(self):
+        self.assertEqual([], os.listdir('.'))
+        tests.write_tree_from_desc('''
+dir: foo
+orphan line
+file: foo/bar.py
+baz
+''')
+        self.assertEqual(['foo'], os.listdir('.'))
+        self.assertEqual(['bar.py'], os.listdir('foo'))
+        self.assertEqual('baz\n', file('foo/bar.py').read())
+
+    def test_empty_file_content(self):
+        tests.write_tree_from_desc('''file: foo''')
+        self.assertEqual('', file('foo').read())
+
+    def test_simple_file_content(self):
+        tests.write_tree_from_desc('''file: foo
+tagada
+''')
+        self.assertEqual('tagada\n', file('foo').read())
+
+    def test_file_content_in_a_dir(self):
+        tests.write_tree_from_desc('''dir: dir
+file: dir/foo
+bar
+''')
+        self.assertEqual('bar\n', file('dir/foo').read())
+
+    def test_simple_symlink_creation(self):
+        tests.write_tree_from_desc('''file: foo
+tagada
+link: foo bar
+''')
+        self.assertEqual('tagada\n', file('foo').read())
+        self.assertEqual('tagada\n', file('bar').read())
+
+    def test_broken_symlink_creation(self):
+        tests.write_tree_from_desc('''link: foo bar
+''')
+        self.assertEqual('foo', os.readlink('bar'))
+
+    def test_invalid_symlink_description_raises(self):
+        e = self.assertRaises(ValueError,
+                              tests.write_tree_from_desc, '''link: foo
+''')
+        self.assertEqual(e.message, 'Invalid link description: foo')

=== modified file 'src/sst/tests/test_xvfb.py'
--- src/sst/tests/test_xvfb.py	2013-01-24 06:16:28 +0000
+++ src/sst/tests/test_xvfb.py	2013-05-14 11:43:29 +0000
@@ -26,7 +26,7 @@
 
 
 from sst import (
-    runtests,
+    cases,
     xvfbdisplay,
 )
 
@@ -49,7 +49,7 @@
         self.assertEquals(orig, os.environ['DISPLAY'])
 
 
-class Headless(runtests.SSTTestCase):
+class Headless(cases.SSTTestCase):
     """A specialized test class for tests around xvfb."""
 
     # We don't use a browser here so disable its use to speed the tests

=== modified file 'src/sst/xvfbdisplay.py'
--- src/sst/xvfbdisplay.py	2013-01-24 06:02:56 +0000
+++ src/sst/xvfbdisplay.py	2013-05-14 11:43:29 +0000
@@ -85,6 +85,21 @@
         os.environ['DISPLAY'] = ':%s' % display_num
 
 
+def use_xvfb_server(test, xvfb=None):
+    """Setup an xvfb server for a given test.
+
+    :param xvfb: An Xvfb object to use. If none is supplied, default values are
+        used to build it.
+
+    :returns: The xvfb server used so tests can use the built one.
+    """
+    if xvfb is None:
+        xvfb = Xvfb()
+    xvfb.start()
+    test.addCleanup(xvfb.stop)
+    return xvfb
+
+
 if __name__ == '__main__':
     # example:
 

=== modified file 'sst-remote'
--- sst-remote	2012-09-27 11:17:49 +0000
+++ sst-remote	2013-05-14 11:43:29 +0000
@@ -1,10 +1,13 @@
 #!/usr/bin/env python
 
-from os.path import dirname, join
-from sys import path
+import os
+import sys
 
 
 if __name__ == "__main__":
-    path.insert(0, join(dirname(__file__), "src"))
+    # We run from sources, we must ensure we won't import from an installed
+    # version so we insert 'src' in front of sys.path
+    cur_dir = os.path.abspath(os.path.dirname(__file__))
+    sys.path.insert(0, os.path.join(cur_dir, 'src'))
     from sst.scripts import remote
     remote.main()

=== modified file 'sst-run'
--- sst-run	2012-09-27 11:17:49 +0000
+++ sst-run	2013-05-14 11:43:29 +0000
@@ -1,10 +1,13 @@
 #!/usr/bin/env python
 
-from os.path import dirname, join
-from sys import path
+import os
+import sys
 
 
 if __name__ == "__main__":
-    path.insert(0, join(dirname(__file__), "src"))
+    # We run from sources, we must ensure we won't import from an installed
+    # version so we insert 'src' in front of sys.path
+    cur_dir = os.path.abspath(os.path.dirname(__file__))
+    sys.path.insert(0, os.path.join(cur_dir, 'src'))
     from sst.scripts import run
     run.main()

=== added file 'test-loader.TODO'
--- test-loader.TODO	1970-01-01 00:00:00 +0000
+++ test-loader.TODO	2013-05-14 11:43:29 +0000
@@ -0,0 +1,39 @@
+* run all tests
+
+  sst test
+
+* run acceptance tests
+
+  sst test -i tests.acceptance
+
+* run unit tests
+
+  sst test -e tests.acceptance
+
+* some test_loader tests are too big, this screams for better helpers so
+   they can be simplified.
+
+* fix the remaining grey areas in the design
+
+There are a few points that smell funny in the current implementation
+including:
+
+- abusing sortTestMethodsUsing
+
+- having to call sortNames in several places
+
+This is probably caused by some less than ideal split of responsability
+between TestLoader and the {File|Dir}Loader classes.
+
+The key points that need to be handled are:
+
+- whether directories are packages or not and as such require to be imported
+
+- if they are imported, respect user overriding
+
+- allowing the user to specify some matching on the base names encountered
+   while walking the tree (or a sub tree). This should remain separate for
+   files and directories (the rules are different).
+
+- how the sub tree is iterated
+