Merge lp:~salgado/offspring/test-runner into lp:offspring

[Guilherme Salgado]

Add a shell script and instructions to run the test suite.

The script uses nose to run the non-django tests and django's standard test
runner for any tests we put under lib/offspring/web/queuemanager.

Currently the new script doesn't expect any arguments but in the future we can
extend it so that users can specify a subset of the tests to run.

[Cody A.W. Somerville (cody-somerville)]

On Thu, Sep 8, 2011 at 3:54 PM, Guilherme Salgado <
<email address hidden>> wrote:
<snip>

> === added directory 'lib/offspring/tests'
> === added file 'test.sh'
> --- test.sh 1970-01-01 00:00:00 +0000
> +++ test.sh 2011-09-08 19:53:46 +0000
> @@ -0,0 +1,6 @@
> +#!/bin/sh
> +
> +# Run all tests, excluding the ones under web.queuemanager.
> +.virtualenv/bin/nosetests -e 'queuemanager.*'
> +# Run the tests under web.queuemanager.
> +./bin/offspring-web test queuemanager
> --settings=offspring.web.settings_test
>

Is there any reason not to just add a new Makefile rule called 'test' to do
this? That way you can even make it make sure everything is setup plus
install nose.

Also, is there any benefit to using nose over updating setup.py so that you
can just run './setup.py test'?

[Guilherme Salgado (salgado)]

Thanks for the review, Cody.

On Wed, 2011-09-14 at 19:18 +0000, Cody A.W. Somerville wrote:
> On Thu, Sep 8, 2011 at 3:54 PM, Guilherme Salgado <
> <email address hidden>> wrote:
> <snip>
>
> > === added directory 'lib/offspring/tests'
> > === added file 'test.sh'
> > --- test.sh 1970-01-01 00:00:00 +0000
> > +++ test.sh 2011-09-08 19:53:46 +0000
> > @@ -0,0 +1,6 @@
> > +#!/bin/sh
> > +
> > +# Run all tests, excluding the ones under web.queuemanager.
> > +.virtualenv/bin/nosetests -e 'queuemanager.*'
> > +# Run the tests under web.queuemanager.
> > +./bin/offspring-web test queuemanager
> > --settings=offspring.web.settings_test
> >
>
> Is there any reason not to just add a new Makefile rule called 'test' to do
> this? That way you can even make it make sure everything is setup plus
> install nose.

Indeed, that's a good idea. I'll do it.

> Also, is there any benefit to using nose over updating setup.py so that you
> can just run './setup.py test'?

Right now the only benefit is that nose does test discovery so we don't
have to enumerate all our tests for setup.py to run them. But it has
several other features that will probably be useful once we have a real
test suite, e.g.

 - package/module level fixtures
 - several options to run just a subset of the tests
 - plugins for code-coverage, dropping into pdb on failure, etc
 - junit output so that we can integrate with subunit/hudson
 - parallel testing, to make use of all those idle cores we have!

[lp:~salgado/offspring/test-runner]

88. By Guilherme Salgado on 2011-09-15

Get rid of the test.sh script and write a couple Makefile targets to run the tests

[Guilherme Salgado (salgado)]

I've just updated this to use new Makefile rules to run the tests, including one extra rule to install nose. However, now that I think of it, shouldn't we install nose as part of the 'develop' rule?

Also, I'm guessing I should also add a nose tarball to lp:~oem-solutions-releng/offspring/offspring-deps/?

[lp:~salgado/offspring/test-runner]

89. By Guilherme Salgado on 2011-09-20

Move installation of nose to the develop Makefile rule

90. By Guilherme Salgado on 2011-09-20

merge trunk

=== modified file 'Makefile'
--- Makefile	2011-09-01 14:42:20 +0000
+++ Makefile	2011-09-20 16:23:23 +0000
@@ -21,6 +21,7 @@
 	[ -f bin/offspring-build ] || ln -s `pwd`/lib/offspring/build/bin/* bin/
 	mkdir -p logs
 	mkdir -p scripts && ln -s --force `pwd`/lib/offspring/build/scripts/* scripts/
+	$(PIP_INSTALL) nose
 
 slave: install-virtualenv requirements/requirements.slave.txt
 	$(PIP_INSTALL) -r requirements/requirements.slave.txt
@@ -42,4 +43,12 @@
 	rm -fr lib/offspring/web/media/js/isomorphic/
 	find lib -name "*.pyc" -delete
 
+test: django-tests non-django-tests
+
+non-django-tests:
+	./.virtualenv/bin/nosetests -e 'queuemanager.*'
+
+django-tests:
+	./bin/offspring-web test queuemanager --settings=offspring.web.settings_test
+
 .PHONY: run develop install-virtualenv

=== added file 'docs/HACKING'
--- docs/HACKING	1970-01-01 00:00:00 +0000
+++ docs/HACKING	2011-09-20 16:23:23 +0000
@@ -0,0 +1,21 @@
+= Running the test suite =
+
+We use nose to run the non-Django tests and the standard Django test runner
+for the tests in lib/offspring/web/queuemanager.
+
+To install nose, run:
+
+  $ make install-nose
+
+After that you can set the OFFSPRING_CONFIGFILE environment variable and run
+the tests with
+
+  $ make test
+
+If you want to run just the Django tests:
+
+  $ make django-tests
+
+Similarly, if you want to run all tests except for the Django ones:
+
+  $ make non-django-tests

=== modified file 'lib/offspring/master/tests.py'
--- lib/offspring/master/tests.py	2011-09-06 18:04:17 +0000
+++ lib/offspring/master/tests.py	2011-09-20 16:23:23 +0000
@@ -2,7 +2,6 @@
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 import email.parser
-import email.utils
 import unittest
 
 from offspring import config
@@ -120,8 +119,3 @@
         from_header = mailer.mails[0][0]
         sender_real_name, sender_addr = email.utils.parseaddr(from_header)
         self.assertEqual(addr, sender_addr)
-
-
-def test_suite():
-    loader = unittest.TestLoader()
-    return loader.loadTestsFromName(__name__)

=== modified file 'lib/offspring/slave/tests.py'
--- lib/offspring/slave/tests.py	2010-11-29 08:27:24 +0000
+++ lib/offspring/slave/tests.py	2011-09-20 16:23:23 +0000
@@ -19,8 +19,3 @@
 
     def test_retrieveState(self):
         self.assertEqual(self.x.getBuilderState(), LexbuilderSlave.STATE_IDLE)
-
-
-def test_suite():
-    loader = unittest.TestLoader()
-    return loader.loadTestsFromName(__name__)

=== added directory 'lib/offspring/tests'