Merge lp:~gmb/zope.testing/make-skips-work-bug-1019275 into lp:~launchpad/zope.testing/3.9.4-fork

Proposed by Graham Binns on 2012-06-29
Status: Merged
Approved by: Brad Crittenden on 2012-06-29
Approved revision: 51
Merged at revision: 47
Proposed branch: lp:~gmb/zope.testing/make-skips-work-bug-1019275
Merge into: lp:~launchpad/zope.testing/3.9.4-fork
Diff against target: 207 lines (+151/-1)
5 files modified
setup.py (+1/-1)
src/zope/testing/testrunner/formatter.py (+9/-0)
src/zope/testing/testrunner/runner.py (+4/-0)
src/zope/testing/testrunner/test_testresult.py (+135/-0)
src/zope/testing/testrunner/tests.py (+2/-0)
To merge this branch: bzr merge lp:~gmb/zope.testing/make-skips-work-bug-1019275
Reviewer Review Type Date Requested Status
Jelmer Vernooij (community) 2012-06-29 Approve on 2012-06-29
Review via email: mp+112762@code.launchpad.net

Commit message

Add an addSkip() method to zope.testing's own version of TestResult. The lack of this method caused skips in python 2.7 to not be recorded in zope.testing-generated subunit streams.

Description of the change

This branch fixes bug 1019275 by adding an addSkip() method to zope.testing.testrunner.runner.TestResult, and by updating the OutputFormatter classes to support test_skip() as a means out outputting data about the skip where possible.

I've added tests to cover the changes; this should help us un-break parallel testing on Python 2.7.

To post a comment you must log in.
Jelmer Vernooij (jelmer) wrote :

As discussed in person:
 * W00t!
 * TestLayer and MockOptions use old style classes
 * Needs empty line under class definitions per PEP8

review: Approve
Launchpad QA Bot (lpqabot) wrote :
Download full text (3.8 KiB)

The attempt to merge lp:~gmb/zope.testing/make-skips-work-bug-1019275 into lp:~launchpad/zope.testing/3.9.4-fork failed. Below is the output from the failed tests.

Creating directory '/home/tarmac/repos/zope.testing/3.9.4-fork/bin'.
Creating directory '/home/tarmac/repos/zope.testing/3.9.4-fork/parts'.
Creating directory '/home/tarmac/repos/zope.testing/3.9.4-fork/eggs'.
Creating directory '/home/tarmac/repos/zope.testing/3.9.4-fork/develop-eggs'.
Getting distribution for 'setuptools'.
Got setuptools 0.6c12dev-r88846.
Generated script '/home/tarmac/repos/zope.testing/3.9.4-fork/bin/buildout'.
Develop: '/home/tarmac/repos/zope.testing/3.9.4-fork/.'
Getting distribution for 'zc.recipe.testrunner'.
Got zc.recipe.testrunner 1.4.0.
Getting distribution for 'z3c.recipe.scripts>=1.0.0'.
Got z3c.recipe.scripts 1.0.1.
Getting distribution for 'zope.testrunner'.
Got zope.testrunner 4.0.4.
Getting distribution for 'zc.recipe.egg>=1.3.0'.
Got zc.recipe.egg 1.3.2.
Getting distribution for 'zope.interface'.
********************************************************************************
WARNING:

        An optional code optimization (C extension) could not be compiled.

        Optimizations for this package will not be available!
()
command 'gcc' failed with exit status 1
********************************************************************************
Got zope.interface 4.0.1.
Getting distribution for 'zope.exceptions'.
Got zope.exceptions 4.0.0.1.
Installing test.
Generated script '/home/tarmac/repos/zope.testing/3.9.4-fork/bin/test'.
Running zope.testing.testrunner.test_testresult.TestLayer tests:
  Set up zope.testing.testrunner.test_testresult.TestLayer in 0.000 seconds.
  Ran 1 tests with 0 failures and 0 errors in 0.000 seconds.
Running zope.testrunner.layer.UnitTests tests:
  Tear down zope.testing.testrunner.test_testresult.TestLayer in 0.000 seconds.
  Set up zope.testrunner.layer.UnitTests in 0.000 seconds.

Error in test test_SubunitOutputFormatter_adds_timestamp_to_skip_if_verbose (zope.testing.testrunner.test_testresult.TestOutputFormatters)
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 327, in run
    testMethod()
  File "/home/tarmac/repos/zope.testing/3.9.4-fork/src/zope/testing/testrunner/test_testresult.py", line 128, in test_SubunitOutputFormatter_adds_timestamp_to_skip_if_verbose
    options=MockOptions(verbose=3), stream=stream)
  File "/home/tarmac/repos/zope.testing/3.9.4-fork/src/zope/testing/testrunner/formatter.py", line 726, in __init__
    raise Exception("Requires subunit 0.0.5 or better")
Exception: Requires subunit 0.0.5 or better

Error in test test_SubunitOutputFormatter_outputs_skip (zope.testing.testrunner.test_testresult.TestOutputFormatters)
Traceback (most recent call last):
  File "/usr/lib/python2.7/unittest/case.py", line 327, in run
    testMethod()
  File "/home/tarmac/repos/zope.testing/3.9.4-fork/src/zope/testing/testrunner/test_testresult.py", line 115, in test_SubunitOutputFormatter_outputs_skip
    options=MockOptions(), stream=stream)
  File "/home/tarmac/repos/zope.testing/3.9.4-fork/src/zope/testing/testrunner/formatter.py", line 726, in __init__
    raise Exce...

Read more...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'setup.py'
2--- setup.py 2012-06-21 16:14:54 +0000
3+++ setup.py 2012-06-29 13:47:18 +0000
4@@ -85,7 +85,7 @@
5
6 setup(
7 name='zope.testing',
8- version = '3.9.4-p15',
9+ version = '3.9.4-p16',
10 url='http://pypi.python.org/pypi/zope.testing',
11 license='ZPL 2.1',
12 description='Zope testing framework, including the testrunner script.',
13
14=== modified file 'src/zope/testing/testrunner/formatter.py'
15--- src/zope/testing/testrunner/formatter.py 2012-06-20 15:05:38 +0000
16+++ src/zope/testing/testrunner/formatter.py 2012-06-29 13:47:18 +0000
17@@ -285,6 +285,10 @@
18
19 sys.stdout.flush()
20
21+ def test_skip(self, test, reason):
22+ """This is a no-op that exists for compatibility reasons."""
23+ pass
24+
25 def test_success(self, test, seconds):
26 """Report that a test was successful.
27
28@@ -910,6 +914,11 @@
29 """Clean up the output state after a collection of tests."""
30 self._stream.flush()
31
32+ def test_skip(self, test, reason):
33+ if self._time_tests:
34+ self._emit_timestamp()
35+ self._subunit.addSkip(test, reason)
36+
37 def test_success(self, test, seconds):
38 if self._time_tests:
39 self._emit_timestamp()
40
41=== modified file 'src/zope/testing/testrunner/runner.py'
42--- src/zope/testing/testrunner/runner.py 2012-06-21 19:40:19 +0000
43+++ src/zope/testing/testrunner/runner.py 2012-06-29 13:47:18 +0000
44@@ -794,6 +794,10 @@
45 self._threads = threading.enumerate()
46 self._start_time = time.time()
47
48+ def addSkip(self, test, reason):
49+ unittest.TestResult.addSkip(self, test, reason)
50+ self.options.output.test_skip(test, reason)
51+
52 def addSuccess(self, test):
53 t = max(time.time() - self._start_time, 0.0)
54 self.options.output.test_success(test, t)
55
56=== added file 'src/zope/testing/testrunner/test_testresult.py'
57--- src/zope/testing/testrunner/test_testresult.py 1970-01-01 00:00:00 +0000
58+++ src/zope/testing/testrunner/test_testresult.py 2012-06-29 13:47:18 +0000
59@@ -0,0 +1,135 @@
60+##############################################################################
61+#
62+# Copyright (c) 2004-2012 Zope Foundation and Contributors.
63+# All Rights Reserved.
64+#
65+# This software is subject to the provisions of the Zope Public License,
66+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
67+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
68+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
69+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
70+# FOR A PARTICULAR PURPOSE.
71+#
72+##############################################################################
73+"""Unit tests for the testrunner's subunit integration.
74+"""
75+
76+__metaclass__ = type
77+
78+import unittest
79+from StringIO import StringIO
80+
81+from zope.testing.testrunner.runner import TestResult
82+from zope.testing.testrunner.formatter import (
83+ OutputFormatter,
84+ SubunitOutputFormatter,
85+ )
86+
87+
88+class MockOutput(StringIO):
89+
90+ def __init__(self):
91+ self.test_skip_called = False
92+
93+ def test_skip(self, test, reason):
94+ self.test_skip_called = True
95+
96+
97+class MockOptions:
98+
99+ def __init__(self, output=None, verbose=None):
100+ self.output = output or MockOutput()
101+ self.verbose = verbose or 1
102+
103+
104+class TestLayer:
105+
106+ def setUp(cls):
107+ pass
108+ setUp = classmethod(setUp)
109+
110+ def tearDown(cls):
111+ pass
112+ tearDown = classmethod(tearDown)
113+
114+ def testSetUp(cls):
115+ pass
116+ testSetUp = classmethod(testSetUp)
117+
118+ def testTearDown(cls):
119+ pass
120+ testTearDown = classmethod(testTearDown)
121+
122+
123+class SampleTestCase(unittest.TestCase):
124+
125+ layer = TestLayer
126+
127+ def test_something(self):
128+ return
129+
130+
131+class TestTestResult(unittest.TestCase):
132+ """Tests for zope.testing's own verison of TestResult."""
133+
134+ def setUp(self):
135+ self.example_suite = (unittest.makeSuite(SampleTestCase))
136+ options = MockOptions()
137+ self.output = options.output
138+ self.result = TestResult(
139+ options, tests=[self.example_suite],
140+ layer_name="zope.testing.testrunner.test_testresult.TestLayer")
141+
142+ def test_addSkip_calls_test_skip_on_output(self):
143+ # TestResult.addSkip() calls the test_skip() method on whatever
144+ # it has as an output formatter.
145+ test = self.example_suite._tests[0]
146+ self.result.addSkip(test, "Testing")
147+ self.assertTrue(self.output.test_skip_called)
148+
149+ def test_addSkip_appends_skipped_test_to_skipped(self):
150+ # TestResult.addSkip() appends the skipped test to the
151+ # TestResult's skipped attribute.
152+ test = self.example_suite._tests[0]
153+ self.result.addSkip(test, "Testing")
154+ self.assertIn((test, "Testing"), self.result.skipped)
155+
156+
157+class TestOutputFormatters(unittest.TestCase):
158+ """Tests for the handling of test_skip by different output formatters."""
159+
160+ def test_OutputFormatter_test_skip_exists(self):
161+ # The test_skip() method of OutputFormatter exists, but it is a
162+ # no-op; it prints nothing.
163+ formatter = OutputFormatter(options=MockOptions())
164+ # We just expect this not to fail.
165+ formatter.test_skip(object(), "blah")
166+
167+ def test_SubunitOutputFormatter_outputs_skip(self):
168+ # SubunitOutputFormatter.test_skip() outputs a skip: $testid
169+ # line to the subunit stream.
170+ example_suite = (unittest.makeSuite(SampleTestCase))
171+ test = example_suite._tests[0]
172+ stream = StringIO()
173+ formatter = SubunitOutputFormatter(
174+ options=MockOptions(), stream=stream)
175+ formatter.test_skip(test, "Testing!")
176+ stream.seek(0)
177+ self.assertEqual(
178+ "skip: %s [\nTesting!\n]\n" % test.id(), stream.read())
179+
180+ def test_SubunitOutputFormatter_adds_timestamp_to_skip_if_verbose(self):
181+ # If the SubunitOutputFormatter is being verbose, it will emit a
182+ # timestamp when it is emitting a skip.
183+ example_suite = (unittest.makeSuite(SampleTestCase))
184+ test = example_suite._tests[0]
185+ stream = StringIO()
186+ formatter = SubunitOutputFormatter(
187+ options=MockOptions(verbose=3), stream=stream)
188+ formatter.test_skip(test, "Testing!")
189+ stream.seek(0)
190+ self.assertTrue(
191+ stream.readline().startswith("time:"),
192+ "SubunitOutputFormatter did not emit a timestamp.")
193+ self.assertEqual(
194+ "skip: %s [\nTesting!\n]\n" % test.id(), stream.read())
195
196=== modified file 'src/zope/testing/testrunner/tests.py'
197--- src/zope/testing/testrunner/tests.py 2012-06-21 20:49:39 +0000
198+++ src/zope/testing/testrunner/tests.py 2012-06-29 13:47:18 +0000
199@@ -200,6 +200,8 @@
200 doctest.DocTestSuite('zope.testing.testrunner.find'),
201 unittest.defaultTestLoader.loadTestsFromName(
202 'zope.testing.testrunner.test_uniqueness'),
203+ unittest.defaultTestLoader.loadTestsFromName(
204+ 'zope.testing.testrunner.test_testresult'),
205 ]
206
207 if sys.platform == 'win32':

Subscribers

People subscribed via source and target branches