Merge lp:~kampka/testtools/test-to-fixture into lp:~testtools-committers/testtools/trunk

[Christian Kampka]

Occasionally I find it useful to have more control over the test within the fixture, e.g. in case something unexpected happens during fixture setup. In order to do so, an instance of the testcase should be available to the fixture.

[Jonathan Lange (jml)]

Interesting!

What sort of thing would you be doing with .test in the fixture?

I'd be interested to see what Rob thinks. He has said before that he likes the way that fixtures can be theoretically used outside of tests. I guess this isn't tainting all fixtures a priori, but it implies that some of *your* fixtures are very much meant to be tightly integrated with testing.

[Christian Kampka (kampka)]

Yes, my fixtures are on occasion very tightly integrated with tests. As a matter of fact, I usually tend to perform most of my test setup within the fixture to make the fixtures behavior consistent when used in multiple test cases/scenarios. In my opinion, this leads to a more complete set of fixtures and a cleaner, easier and more straight forward way of writing tests, but it also means of course that you need to be able to interact with your test from the fixture and vice versa, eg. fail / skip it for some reasons, perform fixture analysis on addOnException calls.
One main reason for this is that some of my fixtures are very expensive to set up because the require parsing large files or because they tap into remote resources, so setting it up from scratch for every test would result in very slow test suite execution. I even find my self in positions where I find it acceptable to share fixtures between tests and even test cases to reduce setup cost, eg. call a slow remote service and use that data on multiple tests. This results in an even tighter integration, eg. "if one test from testcase.id() fails, fail all others" or "if fixture.setUp() fails, skip all tests".

[Jonathan Lange (jml)]

Ahh ok. Doing most of test set up with fixtures is a great idea, and I'm very glad they are working out so well for you. However, I'm not 100% convinced that you need to interact with your test from the fixture. Or, if it is needed, I'd like to know more about the use cases to make a more informed decision about the API.

Can you give me an example of the fail / skip case? That sounds like it might be something worth putting into testtools directly.

As for fixture analysis, what do you need to do that's not provided by the getDetails hook?

Sharing fixtures is great. I think Rob has a project planned for doing it in a clean, easy way. I wonder if this will convince him to write something up and let other folk with more spare hacking time get onto it. :)

[Christian Kampka (kampka)]

I've put together an imaginary example of how I usually interface a fixture with a test, very much simplified of course, but it should show you the general idea. This could probably be done using the current API as well, but I think this is the cleanest way to do it.
http://pastebin.ubuntu.com/629806/

If you or Rob are planing something official on shared fixtures, I'd be happy to contribute my 2 cents of ideas and of course some code if I can :)
My current solution is very much not usable by anyone else but due to an inflexible API that's tailored to my needs, so I'd love to see something more usable.

[Christian Kampka (kampka)]

Is there anything I can do to get this request moving?
I'd like to cross this off my list :)

[Jonathan Lange (jml)]

On Tue, Feb 7, 2012 at 5:44 PM, Christian Kampka <email address hidden> wrote:
> Is there anything I can do to get this request moving?
> I'd like to cross this off my list :)

I'm sorry I've let it languish. I'm a mess of uncertainty about the whole thing.

Will aim to get back to you on it later on today or early tomorrow.

jml

[Jonathan Lange (jml)]

Hi Christian,

My apologies again about the delay. I'm a horrible person.

The fixture API in testtools is very closely aligned with the fixture API in lp:python-fixtures. TestCase implements the Fixture interface, and that's a deliberate thing. Fixtures have their own thing for using (i.e. depending on) other fixtures, which is great.

Just blatting the 'test' attribute is, imho, the wrong thing to do. I would prefer to see something in the Fixture API for setting the parent fixture. ("parent" for want of a better word.) This could be either an attribute or a method call.

On Monday, I'll chat with lifeless, the python-fixtures maintainer, to see what his thoughts are.

I know it's a pain to submit a patch, wait ages, think it's been forgotten only to have it receive a negative review. It's happened to me, and I am very sorry for being the person who has done it to someone else.

I genuinely appreciate your contributions to testtools and hope that this rough patch doesn't sour you to the whole project.

Thanks,
jml

review: Needs Fixing

Unmerged revisions

193. By Christian Kampka on 2011-06-17

* attach test to fixture before running setUp

=== modified file 'testtools/testcase.py'
--- testtools/testcase.py	2011-05-27 17:00:32 +0000
+++ testtools/testcase.py	2011-06-17 16:23:24 +0000
@@ -549,6 +549,7 @@
            it.
         """
         try:
+            fixture.test = self
             fixture.setUp()
         except:
             gather_details(fixture, self)

=== modified file 'testtools/tests/test_fixturesupport.py'
--- testtools/tests/test_fixturesupport.py	2011-05-25 15:03:35 +0000
+++ testtools/tests/test_fixturesupport.py	2011-06-17 16:23:24 +0000
@@ -110,6 +110,10 @@
         self.assertEqual(['content', 'traceback'], sorted(details))
         self.assertEqual('foobar', ''.join(details['content'].iter_text()))
 
+    def test_attacheTestToFixture(self):
+        fixture = LoggingFixture()
+        self.useFixture(fixture)
+        self.assertEqual(fixture.test, self)
 
 def test_suite():
     from unittest import TestLoader