Merge lp:~lifeless/subunit/streamresult into lp:~subunit/subunit/trunk
- streamresult
- Merge into trunk
Proposed by
Robert Collins
Status: | Merged |
---|---|
Merged at revision: | 205 |
Proposed branch: | lp:~lifeless/subunit/streamresult |
Merge into: | lp:~subunit/subunit/trunk |
Diff against target: |
3262 lines (+1848/-501) 25 files modified
Makefile.am (+2/-0) NEWS (+35/-0) README (+252/-14) filters/subunit-1to2 (+42/-0) filters/subunit-2to1 (+46/-0) filters/subunit-filter (+14/-8) filters/subunit-ls (+21/-10) filters/subunit-notify (+5/-1) filters/subunit-stats (+12/-21) filters/subunit2csv (+4/-1) filters/subunit2gtk (+29/-48) filters/subunit2junitxml (+6/-1) filters/subunit2pyunit (+15/-6) python/subunit/__init__.py (+57/-47) python/subunit/filters.py (+75/-15) python/subunit/run.py (+39/-3) python/subunit/test_results.py (+58/-7) python/subunit/tests/__init__.py (+2/-0) python/subunit/tests/test_run.py (+22/-14) python/subunit/tests/test_subunit_filter.py (+43/-61) python/subunit/tests/test_subunit_tags.py (+31/-30) python/subunit/tests/test_tap2subunit.py (+162/-214) python/subunit/tests/test_test_protocol2.py (+415/-0) python/subunit/v2.py (+458/-0) setup.py (+3/-0) |
To merge this branch: | bzr merge lp:~lifeless/subunit/streamresult |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Subunit Developers | Pending | ||
Review via email: mp+151442@code.launchpad.net |
Commit message
Description of the change
Woo! Overlapping concurrent tests with built in enumeration and twice as fast parsing.
To post a comment you must log in.
- 203. By Robert Collins
-
Fixes from getting testrepository running with v2.
- 204. By Robert Collins
-
* ``subunit.run`` now replaces sys.stdout to ensure that stdout is unbuffered
- without this pdb output is not reliably visible when stdout is a pipe
as it usually is. (Robert Collins) - 205. By Robert Collins
-
Switch to variable length encoded integers.
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === modified file 'Makefile.am' |
2 | --- Makefile.am 2012-12-17 07:32:51 +0000 |
3 | +++ Makefile.am 2013-03-31 05:51:20 +0000 |
4 | @@ -47,6 +47,8 @@ |
5 | include_subunitdir = $(includedir)/subunit |
6 | |
7 | dist_bin_SCRIPTS = \ |
8 | + filters/subunit-1to2 \ |
9 | + filters/subunit-2to1 \ |
10 | filters/subunit-filter \ |
11 | filters/subunit-ls \ |
12 | filters/subunit-notify \ |
13 | |
14 | === modified file 'NEWS' |
15 | --- NEWS 2013-02-07 11:33:11 +0000 |
16 | +++ NEWS 2013-03-31 05:51:20 +0000 |
17 | @@ -5,6 +5,41 @@ |
18 | NEXT (In development) |
19 | --------------------- |
20 | |
21 | +v2 protocol draft included in this release. The v2 protocol trades off human |
22 | +readability for a massive improvement in robustness, the ability to represent |
23 | +concurrent tests in a single stream, cheaper parsing, and that provides |
24 | +significantly better in-line debugging support and structured forwarding |
25 | +of non-test data (such as stdout or stdin data). |
26 | + |
27 | +This change includes two new filters (subunit-1to2 and subunit-2to1). Use |
28 | +these filters to convert old streams to v2 and convert v2 streams to v1. |
29 | + |
30 | +All the other filters now only parse and emit v2 streams. V2 is still in |
31 | +draft format, so if you want to delay and wait for v2 to be finalised, you |
32 | +should use subunit-2to1 before any serialisation steps take place. |
33 | +With the ability to encapsulate multiple non-test streams, another significant |
34 | +cange is that filters which emit subunit now encapsulate any non-subunit they |
35 | +encounter, labelling it 'stdout'. This permits multiplexing such streams and |
36 | +detangling the stdout streams from each input. |
37 | + |
38 | +The subunit libraries (Python etc) have not changed their behaviour: they |
39 | +still emit v1 from their existing API calls. New API's are being added |
40 | +and applications should migrate once their language has those API's available. |
41 | + |
42 | +IMPROVEMENTS |
43 | +~~~~~~~~~~~~ |
44 | + |
45 | +* ``subunit.run`` now replaces sys.stdout to ensure that stdout is unbuffered |
46 | + - without this pdb output is not reliably visible when stdout is a pipe |
47 | + as it usually is. (Robert Collins) |
48 | + |
49 | +* v2 protocol draft included in this release. |
50 | + (Robert Collins) |
51 | + |
52 | +* Two new Python classes -- ``StreamResultToBytes`` and |
53 | + ``ByteStreamToStreamResult`` handle v2 generation and parsing. |
54 | + (Robert Collins) |
55 | + |
56 | 0.0.10 |
57 | ------ |
58 | |
59 | |
60 | === modified file 'README' |
61 | --- README 2013-02-07 11:33:11 +0000 |
62 | +++ README 2013-03-31 05:51:20 +0000 |
63 | @@ -21,9 +21,26 @@ |
64 | Subunit |
65 | ------- |
66 | |
67 | -Subunit is a streaming protocol for test results. The protocol is human |
68 | -readable and easily generated and parsed. By design all the components of |
69 | -the protocol conceptually fit into the xUnit TestCase->TestResult interaction. |
70 | +Subunit is a streaming protocol for test results. |
71 | + |
72 | +There are two major revisions of the protocol. Version 1 was trivially human |
73 | +readable but had significant defects as far as highly parallel testing was |
74 | +concerned - it had no room for doing discovery and execution in parallel, |
75 | +required substantial buffering when multiplexing and was fragile - a corrupt |
76 | +byte could cause an entire stream to be misparsed. Version 1.1 added |
77 | +encapsulation of binary streams which mitigated some of the issues but the |
78 | +core remained. |
79 | + |
80 | +Version 2 shares many of the good characteristics of Version 1 - it can be |
81 | +embedded into a regular text stream (e.g. from a build system) and it still |
82 | +models xUnit style test execution. It also fixes many of the issues with |
83 | +Version 1 - Version 2 can be multiplexed without excessive buffering (in |
84 | +time or space), it has a well defined recovery mechanism for dealing with |
85 | +corrupted streams (e.g. where two processes write to the same stream |
86 | +concurrently, or where the stream generator suffers a bug). |
87 | + |
88 | +More details on both protocol version s can be found in the 'Protocol' section |
89 | +of this document. |
90 | |
91 | Subunit comes with command line filters to process a subunit stream and |
92 | language bindings for python, C, C++ and shell. Bindings are easy to write |
93 | @@ -32,11 +49,12 @@ |
94 | A number of useful things can be done easily with subunit: |
95 | * Test aggregation: Tests run separately can be combined and then |
96 | reported/displayed together. For instance, tests from different languages |
97 | - can be shown as a seamless whole. |
98 | + can be shown as a seamless whole, and tests running on multiple machines |
99 | + can be aggregated into a single stream through a multiplexer. |
100 | * Test archiving: A test run may be recorded and replayed later. |
101 | * Test isolation: Tests that may crash or otherwise interact badly with each |
102 | other can be run seperately and then aggregated, rather than interfering |
103 | - with each other. |
104 | + with each other or requiring an adhoc test->runner reporting protocol. |
105 | * Grid testing: subunit can act as the necessary serialisation and |
106 | deserialiation to get test runs on distributed machines to be reported in |
107 | real time. |
108 | @@ -68,20 +86,20 @@ |
109 | in python and there are facilities for using Subunit to increase test isolation |
110 | seamlessly within a test suite. |
111 | |
112 | -One simple way to run an existing python test suite and have it output subunit |
113 | -is the module ``subunit.run``:: |
114 | +The most common way is to run an existing python test suite and have it output |
115 | +subunit via the ``subunit.run`` module:: |
116 | |
117 | $ python -m subunit.run mypackage.tests.test_suite |
118 | |
119 | For more information on the Python support Subunit offers , please see |
120 | -``pydoc subunit``, or the source in ``python/subunit/__init__.py`` |
121 | +``pydoc subunit``, or the source in ``python/subunit/`` |
122 | |
123 | C |
124 | = |
125 | |
126 | -Subunit has C bindings to emit the protocol, and comes with a patch for 'check' |
127 | -which has been nominally accepted by the 'check' developers. See 'c/README' for |
128 | -more details. |
129 | +Subunit has C bindings to emit the protocol. The 'check' C unit testing project |
130 | +has included subunit support in their project for some years now. See |
131 | +'c/README' for more details. |
132 | |
133 | C++ |
134 | === |
135 | @@ -92,9 +110,13 @@ |
136 | shell |
137 | ===== |
138 | |
139 | -Similar to C, the shell bindings consist of simple functions to output protocol |
140 | -elements, and a patch for adding subunit output to the 'ShUnit' shell test |
141 | -runner. See 'shell/README' for details. |
142 | +There are two sets of shell tools. There are filters, which accept a subunit |
143 | +stream on stdin and output processed data (or a transformed stream) on stdout. |
144 | + |
145 | +Then there are unittest facilities similar to those for C : shell bindings |
146 | +consisting of simple functions to output protocol elements, and a patch for |
147 | +adding subunit output to the 'ShUnit' shell test runner. See 'shell/README' for |
148 | +details. |
149 | |
150 | Filter recipes |
151 | -------------- |
152 | @@ -104,9 +126,225 @@ |
153 | subunit-filter --without 'AttributeError.*flavor' |
154 | |
155 | |
156 | +The xUnit test model |
157 | +-------------------- |
158 | + |
159 | +Subunit implements a slightly modified xUnit test model. The stock standard |
160 | +model is that there are tests, which have an id(), can be run, and when run |
161 | +start, emit an outcome (like success or failure) and then finish. |
162 | + |
163 | +Subunit extends this with the idea of test enumeration (find out about tests |
164 | +a runner has without running them), tags (allow users to describe tests in |
165 | +ways the test framework doesn't apply any semantic value to), file attachments |
166 | +(allow arbitrary data to make analysing a failure easy) and timestamps. |
167 | + |
168 | The protocol |
169 | ------------ |
170 | |
171 | +Version 2, or v2 is new and still under development, but is intended to |
172 | +supercede version 1 in the very near future. Subunit's bundled tools accept |
173 | +only version 2 and only emit version 2, but the new filters subunit-1to2 and |
174 | +subunit-2to1 can be used to interoperate with older third party libraries. |
175 | + |
176 | +Version 2 |
177 | +========= |
178 | + |
179 | +Version 2 is a binary protocol consisting of independent packets that can be |
180 | +embedded in the output from tools like make - as long as each packet has no |
181 | +other bytes mixed in with it (which 'make -j N>1' has a tendency of doing). |
182 | +Version 2 is currently in draft form, and early adopters should be willing |
183 | +to either discard stored results (if protocol changes are made), or bulk |
184 | +convert them back to v1 and then to a newer edition of v2. |
185 | + |
186 | +The protocol synchronises at the start of the stream, after a packet, or |
187 | +after any 0x0A byte. That is, a subunit v2 packet starts after a newline or |
188 | +directly after the end of the prior packet. |
189 | + |
190 | +Subunit is intended to be transported over a reliable streaming protocol such |
191 | +as TCP. As such it does not concern itself with out of order delivery of |
192 | +packets. However, because of the possibility of corruption due to either |
193 | +bugs in the sender, or due to mixed up data from concurrent writes to the same |
194 | +fd when being embedded, subunit strives to recover reasonably gracefully from |
195 | +damaged data. |
196 | + |
197 | +A key design goal for Subunit version 2 is to allow processing and multiplexing |
198 | +without forcing buffering for semantic correctness, as buffering tends to hide |
199 | +hung or otherwise misbehaving tests. That said, limited time based buffering |
200 | +for network efficiency is a good idea - this is ultimately implementator |
201 | +choice. Line buffering is also discouraged for subunit streams, as dropping |
202 | +into a debugger or other tool may require interactive traffic even if line |
203 | +buffering would not otherwise be a problem. |
204 | + |
205 | +In version two there are two conceptual events - a test status event and a file |
206 | +attachment event. Events may have timestamps, and the path of multiplexers that |
207 | +an event is routed through is recorded to permit sending actions back to the |
208 | +source (such as new tests to run or stdin for driving debuggers and other |
209 | +interactive input). Test status events are used to enumerate tests, to report |
210 | +tests and test helpers as they run. Tests may have tags, used to allow |
211 | +tunnelling extra meanings through subunit without requiring parsing of |
212 | +arbitrary file attachments. Things that are not standalone tests get marked |
213 | +as such by setting the 'Runnable' flag to false. (For instance, individual |
214 | +assertions in TAP are not runnable tests, only the top level TAP test script |
215 | +is runnable). |
216 | + |
217 | +File attachments are used to provide rich detail about the nature of a failure. |
218 | +File attachments can also be used to encapsulate stdout and stderr both during |
219 | +and outside tests. |
220 | + |
221 | +Most numbers are stored in network byte order - Most Significant Byte first |
222 | +encoded using a variation of http://www.dlugosz.com/ZIP2/VLI.html. The first |
223 | +byte's top 2 high order bits encode the total number of octets in the number. |
224 | +This encoding can encode values from 0 to 2**30-1, enough to encode a |
225 | +nanosecond. Numbers that are not variable length encoded are still stored in |
226 | +MSB order. |
227 | + |
228 | + prefix octets max max |
229 | ++-------+--------+---------+------------+ |
230 | +| 00 | 1 | 2**6-1 | 63 | |
231 | +| 01 | 2 | 2**14-1 | 16383 | |
232 | +| 10 | 3 | 2**22-1 | 4194303 | |
233 | +| 11 | 4 | 2**30-1 | 1073741823 | |
234 | ++-------+--------+---------+------------+ |
235 | + |
236 | +All variable length elements of the packet are stored with a length prefix |
237 | +number allowing them to be skipped over for consumers that don't need to |
238 | +interpret them. |
239 | + |
240 | +UTF-8 strings are with no terminating NUL and should not have any embedded NULs |
241 | +(implementations SHOULD validate any such strings that they process and take |
242 | +some remedial action (such as discarding the packet as corrupt). |
243 | + |
244 | +In short the structure of a packet is: |
245 | +PACKET := SIGNATURE FLAGS PACKET_LENGTH TIMESTAMP? TESTID? TAGS? MIME? |
246 | + FILECONTENT? ROUTING_CODE? CRC32 |
247 | + |
248 | +In more detail... |
249 | + |
250 | +Packets are identified by a single byte signature - 0xB3, which is never legal |
251 | +in a UTF-8 stream as the first byte of a character. 0xB3 starts with the first |
252 | +bit set and the second not, which is the UTF-8 signature for a continuation |
253 | +byte. 0xB3 was chosen as 0x73 ('s' in ASCII') with the top two bits replaced by |
254 | +the 1 and 0 for a continuation byte. |
255 | + |
256 | +If subunit packets are being embedded in a non-UTF-8 text stream, where 0x73 is |
257 | +a legal character, consider either recoding the text to UTF-8, or using |
258 | +subunit's 'file' packets to embed the text stream in subunit, rather than the |
259 | +other way around. |
260 | + |
261 | +Following the signature byte comes a 16-bit flags field, which includes a |
262 | +4-bit version field - if the version is not 0x2 then the packet cannot be |
263 | +read. It is recommended to signal an error at this point (e.g. by emitting |
264 | +a synthetic error packet and returning to the top level loop to look for |
265 | +new packets, or exiting with an error). If recovery is desired, treat the |
266 | +packet signature as an opaque byte and scan for a new synchronisation point. |
267 | +NB: Subunit V1 and V2 packets may legitimately included 0xB3 internally, |
268 | +as they are an 8-bit safe container format, so recovery from this situation |
269 | +may involve an arbitrary number of false positives until an actual packet |
270 | +is encountered : and even then it may still be false, failing after passing |
271 | +the version check due to coincidence. |
272 | + |
273 | +Flags are stored in network byte order too. |
274 | ++-------------------------+------------------------+ |
275 | +| High byte | Low byte | |
276 | +| 15 14 13 12 11 10 9 8 | 7 6 5 4 3 2 1 0 | |
277 | +| VERSION |feature bits| | |
278 | ++------------+------------+------------------------+ |
279 | + |
280 | +Valid version values are: |
281 | +0x2 - version 2 |
282 | + |
283 | +Feature bits: |
284 | +Bit 11 - mask 0x0800 - Test id present. |
285 | +Bit 10 - mask 0x0400 - Routing code present. |
286 | +Bit 9 - mask 0x0200 - Timestamp present. |
287 | +Bit 8 - mask 0x0100 - Test is 'runnable'. |
288 | +Bit 7 - mask 0x0080 - Tags are present. |
289 | +Bit 6 - mask 0x0040 - File content is present. |
290 | +Bit 5 - mask 0x0020 - File MIME type is present. |
291 | +Bit 4 - mask 0x0010 - EOF marker. |
292 | +Bit 3 - mask 0x0008 - Must be zero in version 2. |
293 | + |
294 | +Test status gets three bits: |
295 | +Bit 2 | Bit 1 | Bit 0 - mask 0x0007 - A test status enum lookup: |
296 | +000 - undefined / no test |
297 | +001 - Enumeration / existence |
298 | +002 - In progress |
299 | +003 - Success |
300 | +004 - Unexpected Success |
301 | +005 - Skipped |
302 | +006 - Failed |
303 | +007 - Expected failure |
304 | + |
305 | +After the flags field is a number field giving the length in bytes for the |
306 | +entire packet including the signature and the checksum. This length must |
307 | +be less than 4MiB - 4194303 bytes. The encoding can obviously record a larger |
308 | +number but one of the goals is to avoid requiring large buffers, or causing |
309 | +large latency in the packet forward/processing pipeline. Larger file |
310 | +attachments can be communicated in multiple packets, and the overhead in such a |
311 | +4MiB packet is approximately 0.2%. |
312 | + |
313 | +The rest of the packet is a series of optional features as specified by the set |
314 | +feature bits in the flags field. When absent they are entirely absent. |
315 | + |
316 | +Forwarding and multiplexing of packets can be done without interpreting the |
317 | +remainder of the packet until the routing code and checksum (which are both at |
318 | +the end of the packet). Additionally, routers can often avoid copying or moving |
319 | +the bulk of the packet, as long as the routing code size increase doesn't force |
320 | +the length encoding to take up a new byte (which will only happen to packets |
321 | +less than or equal to 16KiB in length) - large packets are very efficient to |
322 | +route. |
323 | + |
324 | +Timestamp when present is a 32 bit unsigned integer for secnods, and a variable |
325 | +length number for nanoseconds, representing UTC time since Unix Epoch in |
326 | +seconds and nanoseconds. |
327 | + |
328 | +Test id when present is a UTF-8 string. The test id should uniquely identify |
329 | +runnable tests such that they can be selected individually. For tests and other |
330 | +actions which cannot be individually run (such as test |
331 | +fixtures/layers/subtests) uniqueness is not required (though being human |
332 | +meaningful is highly recommended). |
333 | + |
334 | +Tags when present is a length prefixed vector of UTF-8 strings, one per tag. |
335 | +There are no restrictions on tag content (other than the restrictions on UTF-8 |
336 | +strings in subunit in general). Tags have no ordering. |
337 | + |
338 | +When a MIME type is present, it defines the MIME type for the file across all |
339 | +packets same file (routing code + testid + name uniquely identifies a file, |
340 | +reset when EOF is flagged). If a file never has a MIME type set, it should be |
341 | +treated as application/octet-stream. |
342 | + |
343 | +File content when present is a UTF-8 string for the name followed by the length |
344 | +in bytes of the content, and then the content octets. |
345 | + |
346 | +If present routing code is a UTF-8 string. The routing code is used to |
347 | +determine which test backend a test was running on when doing data analysis, |
348 | +and to route stdin to the test process if interaction is required. |
349 | + |
350 | +Multiplexers SHOULD add a routing code if none is present, and prefix any |
351 | +existing routing code with a routing code ('/' separated) if one is already |
352 | +present. For example, a multiplexer might label each stream it is multiplexing |
353 | +with a simple ordinal ('0', '1' etc), and given an incoming packet with route |
354 | +code '3' from stream '0' would adjust the route code when forwarding the packet |
355 | +to be '0/3'. |
356 | + |
357 | +Following the end of the packet is a CRC-32 checksum of the contents of the |
358 | +packet including the signature. |
359 | + |
360 | +Example packets |
361 | +~~~~~~~~~~~~~~~ |
362 | + |
363 | +Trivial test "foo" enumeration packet, with test id, runnable set, |
364 | +status=enumeration. Spaces below are to visually break up signature / flags / |
365 | +length / testid / crc32 |
366 | + |
367 | +b3 2901 0c 03666f6f 08555f1b |
368 | + |
369 | + |
370 | +Version 1 (and 1.1) |
371 | +=================== |
372 | + |
373 | +Version 1 (and 1.1) are mostly human readable protocols. |
374 | + |
375 | Sample subunit wire contents |
376 | ---------------------------- |
377 | |
378 | |
379 | === added file 'filters/subunit-1to2' |
380 | --- filters/subunit-1to2 1970-01-01 00:00:00 +0000 |
381 | +++ filters/subunit-1to2 2013-03-31 05:51:20 +0000 |
382 | @@ -0,0 +1,42 @@ |
383 | +#!/usr/bin/env python |
384 | +# subunit: extensions to python unittest to get test results from subprocesses. |
385 | +# Copyright (C) 2013 Robert Collins <robertc@robertcollins.net> |
386 | +# |
387 | +# Licensed under either the Apache License, Version 2.0 or the BSD 3-clause |
388 | +# license at the users choice. A copy of both licenses are available in the |
389 | +# project source as Apache-2.0 and BSD. You may not use this file except in |
390 | +# compliance with one of these two licences. |
391 | +# |
392 | +# Unless required by applicable law or agreed to in writing, software |
393 | +# distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT |
394 | +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the |
395 | +# license you chose for the specific language governing permissions and |
396 | +# limitations under that license. |
397 | +# |
398 | + |
399 | +"""Convert a version 1 subunit stream to version 2 stream.""" |
400 | + |
401 | +from optparse import OptionParser |
402 | +import sys |
403 | + |
404 | +from testtools import ExtendedToStreamDecorator |
405 | + |
406 | +from subunit import StreamResultToBytes |
407 | +from subunit.filters import run_tests_from_stream |
408 | + |
409 | + |
410 | +def make_options(description): |
411 | + parser = OptionParser(description=__doc__) |
412 | + return parser |
413 | + |
414 | + |
415 | +def main(): |
416 | + parser = make_options(__doc__) |
417 | + (options, args) = parser.parse_args() |
418 | + run_tests_from_stream(sys.stdin, |
419 | + ExtendedToStreamDecorator(StreamResultToBytes(sys.stdout))) |
420 | + sys.exit(0) |
421 | + |
422 | + |
423 | +if __name__ == '__main__': |
424 | + main() |
425 | |
426 | === added file 'filters/subunit-2to1' |
427 | --- filters/subunit-2to1 1970-01-01 00:00:00 +0000 |
428 | +++ filters/subunit-2to1 2013-03-31 05:51:20 +0000 |
429 | @@ -0,0 +1,46 @@ |
430 | +#!/usr/bin/env python |
431 | +# subunit: extensions to python unittest to get test results from subprocesses. |
432 | +# Copyright (C) 2013 Robert Collins <robertc@robertcollins.net> |
433 | +# |
434 | +# Licensed under either the Apache License, Version 2.0 or the BSD 3-clause |
435 | +# license at the users choice. A copy of both licenses are available in the |
436 | +# project source as Apache-2.0 and BSD. You may not use this file except in |
437 | +# compliance with one of these two licences. |
438 | +# |
439 | +# Unless required by applicable law or agreed to in writing, software |
440 | +# distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT |
441 | +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the |
442 | +# license you chose for the specific language governing permissions and |
443 | +# limitations under that license. |
444 | +# |
445 | + |
446 | +"""Convert a version 2 subunit stream to a version 1 stream.""" |
447 | + |
448 | +from optparse import OptionParser |
449 | +import sys |
450 | + |
451 | +from testtools import StreamToExtendedDecorator |
452 | + |
453 | +from subunit import ByteStreamToStreamResult, TestProtocolClient |
454 | +from subunit.filters import run_tests_from_stream |
455 | + |
456 | + |
457 | +def make_options(description): |
458 | + parser = OptionParser(description=__doc__) |
459 | + return parser |
460 | + |
461 | + |
462 | +def main(): |
463 | + parser = make_options(__doc__) |
464 | + (options, args) = parser.parse_args() |
465 | + case = ByteStreamToStreamResult(sys.stdin, non_subunit_name='stdout') |
466 | + result = StreamToExtendedDecorator(TestProtocolClient(sys.stdout)) |
467 | + # What about stdout chunks? |
468 | + result.startTestRun() |
469 | + case.run(result) |
470 | + result.stopTestRun() |
471 | + sys.exit(0) |
472 | + |
473 | + |
474 | +if __name__ == '__main__': |
475 | + main() |
476 | |
477 | === modified file 'filters/subunit-filter' |
478 | --- filters/subunit-filter 2012-05-03 08:18:01 +0000 |
479 | +++ filters/subunit-filter 2013-03-31 05:51:20 +0000 |
480 | @@ -1,6 +1,6 @@ |
481 | #!/usr/bin/env python |
482 | # subunit: extensions to python unittest to get test results from subprocesses. |
483 | -# Copyright (C) 2008 Robert Collins <robertc@robertcollins.net> |
484 | +# Copyright (C) 200-2013 Robert Collins <robertc@robertcollins.net> |
485 | # (C) 2009 Martin Pool |
486 | # |
487 | # Licensed under either the Apache License, Version 2.0 or the BSD 3-clause |
488 | @@ -30,10 +30,12 @@ |
489 | import sys |
490 | import re |
491 | |
492 | +from testtools import ExtendedToStreamDecorator, StreamToExtendedDecorator |
493 | + |
494 | from subunit import ( |
495 | DiscardStream, |
496 | ProtocolTestCase, |
497 | - TestProtocolClient, |
498 | + StreamResultToBytes, |
499 | read_test_list, |
500 | ) |
501 | from subunit.filters import filter_by_result |
502 | @@ -55,9 +57,11 @@ |
503 | parser.add_option("-f", "--no-failure", action="store_true", |
504 | help="exclude failures", dest="failure") |
505 | parser.add_option("--passthrough", action="store_false", |
506 | - help="Show all non subunit input.", default=False, dest="no_passthrough") |
507 | + help="Forward non-subunit input as 'stdout'.", default=False, |
508 | + dest="no_passthrough") |
509 | parser.add_option("--no-passthrough", action="store_true", |
510 | - help="Hide all non subunit input.", default=False, dest="no_passthrough") |
511 | + help="Discard all non subunit input.", default=False, |
512 | + dest="no_passthrough") |
513 | parser.add_option("-s", "--success", action="store_false", |
514 | help="include successes", dest="success") |
515 | parser.add_option("--no-success", action="store_true", |
516 | @@ -126,15 +130,16 @@ |
517 | fixup_expected_failures = set() |
518 | for path in options.fixup_expected_failures or (): |
519 | fixup_expected_failures.update(read_test_list(path)) |
520 | - return TestResultFilter( |
521 | - TestProtocolClient(output), |
522 | + return StreamToExtendedDecorator(TestResultFilter( |
523 | + ExtendedToStreamDecorator( |
524 | + StreamResultToBytes(output)), |
525 | filter_error=options.error, |
526 | filter_failure=options.failure, |
527 | filter_success=options.success, |
528 | filter_skip=options.skip, |
529 | filter_xfail=options.xfail, |
530 | filter_predicate=predicate, |
531 | - fixup_expected_failures=fixup_expected_failures) |
532 | + fixup_expected_failures=fixup_expected_failures)) |
533 | |
534 | |
535 | def main(): |
536 | @@ -150,7 +155,8 @@ |
537 | lambda output_to: _make_result(sys.stdout, options, filter_predicate), |
538 | output_path=None, |
539 | passthrough=(not options.no_passthrough), |
540 | - forward=False) |
541 | + forward=False, |
542 | + protocol_version=2) |
543 | sys.exit(0) |
544 | |
545 | |
546 | |
547 | === modified file 'filters/subunit-ls' |
548 | --- filters/subunit-ls 2011-05-23 09:57:58 +0000 |
549 | +++ filters/subunit-ls 2013-03-31 05:51:20 +0000 |
550 | @@ -19,9 +19,14 @@ |
551 | from optparse import OptionParser |
552 | import sys |
553 | |
554 | -from subunit import DiscardStream, ProtocolTestCase |
555 | +from testtools import ( |
556 | + CopyStreamResult, StreamToExtendedDecorator, StreamResultRouter, |
557 | + StreamSummary) |
558 | + |
559 | +from subunit import ByteStreamToStreamResult |
560 | +from subunit.filters import run_tests_from_stream |
561 | from subunit.test_results import ( |
562 | - AutoTimingTestResultDecorator, |
563 | + CatFiles, |
564 | TestIdPrintingResult, |
565 | ) |
566 | |
567 | @@ -30,18 +35,24 @@ |
568 | parser.add_option("--times", action="store_true", |
569 | help="list the time each test took (requires a timestamped stream)", |
570 | default=False) |
571 | +parser.add_option("--exists", action="store_true", |
572 | + help="list tests that are reported as existing (as well as ran)", |
573 | + default=False) |
574 | parser.add_option("--no-passthrough", action="store_true", |
575 | help="Hide all non subunit input.", default=False, dest="no_passthrough") |
576 | (options, args) = parser.parse_args() |
577 | -result = AutoTimingTestResultDecorator( |
578 | - TestIdPrintingResult(sys.stdout, options.times)) |
579 | -if options.no_passthrough: |
580 | - passthrough_stream = DiscardStream() |
581 | -else: |
582 | - passthrough_stream = None |
583 | -test = ProtocolTestCase(sys.stdin, passthrough=passthrough_stream) |
584 | +test = ByteStreamToStreamResult(sys.stdin, non_subunit_name="stdout") |
585 | +result = TestIdPrintingResult(sys.stdout, options.times, options.exists) |
586 | +if not options.no_passthrough: |
587 | + result = StreamResultRouter(result) |
588 | + cat = CatFiles(sys.stdout) |
589 | + result.map(cat, 'test_id', test_id=None) |
590 | +summary = StreamSummary() |
591 | +result = CopyStreamResult([result, summary]) |
592 | +result.startTestRun() |
593 | test.run(result) |
594 | -if result.wasSuccessful(): |
595 | +result.stopTestRun() |
596 | +if summary.wasSuccessful(): |
597 | exit_code = 0 |
598 | else: |
599 | exit_code = 1 |
600 | |
601 | === modified file 'filters/subunit-notify' |
602 | --- filters/subunit-notify 2012-03-27 11:17:37 +0000 |
603 | +++ filters/subunit-notify 2013-03-31 05:51:20 +0000 |
604 | @@ -19,6 +19,7 @@ |
605 | import pygtk |
606 | pygtk.require('2.0') |
607 | import pynotify |
608 | +from testtools import StreamToExtendedDecorator |
609 | |
610 | from subunit import TestResultStats |
611 | from subunit.filters import run_filter_script |
612 | @@ -28,6 +29,7 @@ |
613 | |
614 | |
615 | def notify_of_result(result): |
616 | + result = result.decorated |
617 | if result.failed_tests > 0: |
618 | summary = "Test run failed" |
619 | else: |
620 | @@ -41,4 +43,6 @@ |
621 | nw.show() |
622 | |
623 | |
624 | -run_filter_script(TestResultStats, __doc__, notify_of_result) |
625 | +run_filter_script( |
626 | + lambda output:StreamToExtendedDecorator(TestResultStats(output)), |
627 | + __doc__, notify_of_result, protocol_version=2) |
628 | |
629 | === modified file 'filters/subunit-stats' |
630 | --- filters/subunit-stats 2009-09-30 12:04:18 +0000 |
631 | +++ filters/subunit-stats 2013-03-31 05:51:20 +0000 |
632 | @@ -16,26 +16,17 @@ |
633 | |
634 | """Filter a subunit stream to get aggregate statistics.""" |
635 | |
636 | -from optparse import OptionParser |
637 | import sys |
638 | -import unittest |
639 | - |
640 | -from subunit import DiscardStream, ProtocolTestCase, TestResultStats |
641 | - |
642 | -parser = OptionParser(description=__doc__) |
643 | -parser.add_option("--no-passthrough", action="store_true", |
644 | - help="Hide all non subunit input.", default=False, dest="no_passthrough") |
645 | -(options, args) = parser.parse_args() |
646 | + |
647 | +from testtools import StreamToExtendedDecorator |
648 | + |
649 | +from subunit import TestResultStats |
650 | +from subunit.filters import run_filter_script |
651 | + |
652 | + |
653 | result = TestResultStats(sys.stdout) |
654 | -if options.no_passthrough: |
655 | - passthrough_stream = DiscardStream() |
656 | -else: |
657 | - passthrough_stream = None |
658 | -test = ProtocolTestCase(sys.stdin, passthrough=passthrough_stream) |
659 | -test.run(result) |
660 | -result.formatStats() |
661 | -if result.wasSuccessful(): |
662 | - exit_code = 0 |
663 | -else: |
664 | - exit_code = 1 |
665 | -sys.exit(exit_code) |
666 | +def show_stats(r): |
667 | + r.decorated.formatStats() |
668 | +run_filter_script( |
669 | + lambda output:StreamToExtendedDecorator(result), |
670 | + __doc__, show_stats, protocol_version=2) |
671 | |
672 | === modified file 'filters/subunit2csv' |
673 | --- filters/subunit2csv 2012-03-27 10:57:51 +0000 |
674 | +++ filters/subunit2csv 2013-03-31 05:51:20 +0000 |
675 | @@ -16,8 +16,11 @@ |
676 | |
677 | """Turn a subunit stream into a CSV""" |
678 | |
679 | +from testtools import StreamToExtendedDecorator |
680 | + |
681 | from subunit.filters import run_filter_script |
682 | from subunit.test_results import CsvResult |
683 | |
684 | |
685 | -run_filter_script(CsvResult, __doc__) |
686 | +run_filter_script(lambda output:StreamToExtendedDecorator(CsvResult(output)), |
687 | + __doc__, protocol_version=2) |
688 | |
689 | === modified file 'filters/subunit2gtk' |
690 | --- filters/subunit2gtk 2010-01-25 15:45:45 +0000 |
691 | +++ filters/subunit2gtk 2013-03-31 05:51:20 +0000 |
692 | @@ -46,17 +46,20 @@ |
693 | """Display a subunit stream in a gtk progress window.""" |
694 | |
695 | import sys |
696 | +import threading |
697 | import unittest |
698 | |
699 | import pygtk |
700 | pygtk.require('2.0') |
701 | import gtk, gtk.gdk, gobject |
702 | |
703 | +from testtools import StreamToExtendedDecorator |
704 | + |
705 | from subunit import ( |
706 | PROGRESS_POP, |
707 | PROGRESS_PUSH, |
708 | PROGRESS_SET, |
709 | - TestProtocolServer, |
710 | + ByteStreamToStreamResult, |
711 | ) |
712 | from subunit.progress_model import ProgressModel |
713 | |
714 | @@ -139,6 +142,9 @@ |
715 | |
716 | def stopTest(self, test): |
717 | super(GTKTestResult, self).stopTest(test) |
718 | + gobject.idle_add(self._stopTest) |
719 | + |
720 | + def _stopTest(self): |
721 | self.progress_model.advance() |
722 | if self.progress_model.width() == 0: |
723 | self.pbar.pulse() |
724 | @@ -153,26 +159,26 @@ |
725 | super(GTKTestResult, self).stopTestRun() |
726 | except AttributeError: |
727 | pass |
728 | - self.pbar.set_text('Finished') |
729 | + gobject.idle_add(self.pbar.set_text, 'Finished') |
730 | |
731 | def addError(self, test, err): |
732 | super(GTKTestResult, self).addError(test, err) |
733 | - self.update_counts() |
734 | + gobject.idle_add(self.update_counts) |
735 | |
736 | def addFailure(self, test, err): |
737 | super(GTKTestResult, self).addFailure(test, err) |
738 | - self.update_counts() |
739 | + gobject.idle_add(self.update_counts) |
740 | |
741 | def addSuccess(self, test): |
742 | super(GTKTestResult, self).addSuccess(test) |
743 | - self.update_counts() |
744 | + gobject.idle_add(self.update_counts) |
745 | |
746 | def addSkip(self, test, reason): |
747 | # addSkip is new in Python 2.7/3.1 |
748 | addSkip = getattr(super(GTKTestResult, self), 'addSkip', None) |
749 | if callable(addSkip): |
750 | addSkip(test, reason) |
751 | - self.update_counts() |
752 | + gobject.idle_add(self.update_counts) |
753 | |
754 | def addExpectedFailure(self, test, err): |
755 | # addExpectedFailure is new in Python 2.7/3.1 |
756 | @@ -180,7 +186,7 @@ |
757 | 'addExpectedFailure', None) |
758 | if callable(addExpectedFailure): |
759 | addExpectedFailure(test, err) |
760 | - self.update_counts() |
761 | + gobject.idle_add(self.update_counts) |
762 | |
763 | def addUnexpectedSuccess(self, test): |
764 | # addUnexpectedSuccess is new in Python 2.7/3.1 |
765 | @@ -188,7 +194,7 @@ |
766 | 'addUnexpectedSuccess', None) |
767 | if callable(addUnexpectedSuccess): |
768 | addUnexpectedSuccess(test) |
769 | - self.update_counts() |
770 | + gobject.idle_add(self.update_counts) |
771 | |
772 | def progress(self, offset, whence): |
773 | if whence == PROGRESS_PUSH: |
774 | @@ -212,47 +218,22 @@ |
775 | self.ok_label.set_text(str(self.testsRun - bad)) |
776 | self.not_ok_label.set_text(str(bad)) |
777 | |
778 | - |
779 | -class GIOProtocolTestCase(object): |
780 | - |
781 | - def __init__(self, stream, result, on_finish): |
782 | - self.stream = stream |
783 | - self.schedule_read() |
784 | - self.hup_id = gobject.io_add_watch(stream, gobject.IO_HUP, self.hup) |
785 | - self.protocol = TestProtocolServer(result) |
786 | - self.on_finish = on_finish |
787 | - |
788 | - def read(self, source, condition, all=False): |
789 | - #NB: \o/ actually blocks |
790 | - line = source.readline() |
791 | - if not line: |
792 | - self.protocol.lostConnection() |
793 | - self.on_finish() |
794 | - return False |
795 | - self.protocol.lineReceived(line) |
796 | - # schedule more IO shortly - if we say we're willing to do it |
797 | - # immediately we starve things. |
798 | - if not all: |
799 | - source_id = gobject.timeout_add(1, self.schedule_read) |
800 | - return False |
801 | - else: |
802 | - return True |
803 | - |
804 | - def schedule_read(self): |
805 | - self.read_id = gobject.io_add_watch(self.stream, gobject.IO_IN, self.read) |
806 | - |
807 | - def hup(self, source, condition): |
808 | - while self.read(source, condition, all=True): pass |
809 | - self.protocol.lostConnection() |
810 | - gobject.source_remove(self.read_id) |
811 | - self.on_finish() |
812 | - return False |
813 | - |
814 | - |
815 | -result = GTKTestResult() |
816 | -test = GIOProtocolTestCase(sys.stdin, result, result.stopTestRun) |
817 | +gobject.threads_init() |
818 | +result = StreamToExtendedDecorator(GTKTestResult()) |
819 | +test = ByteStreamToStreamResult(sys.stdin, non_subunit_name='stdout') |
820 | +# Get setup |
821 | +while gtk.events_pending(): |
822 | + gtk.main_iteration() |
823 | +# Start IO |
824 | +def run_and_finish(): |
825 | + test.run(result) |
826 | + result.stopTestRun() |
827 | +t = threading.Thread(target=run_and_finish) |
828 | +t.daemon = True |
829 | +result.startTestRun() |
830 | +t.start() |
831 | gtk.main() |
832 | -if result.wasSuccessful(): |
833 | +if result.decorated.wasSuccessful(): |
834 | exit_code = 0 |
835 | else: |
836 | exit_code = 1 |
837 | |
838 | === modified file 'filters/subunit2junitxml' |
839 | --- filters/subunit2junitxml 2012-03-27 10:57:51 +0000 |
840 | +++ filters/subunit2junitxml 2013-03-31 05:51:20 +0000 |
841 | @@ -18,6 +18,9 @@ |
842 | |
843 | |
844 | import sys |
845 | + |
846 | +from testtools import StreamToExtendedDecorator |
847 | + |
848 | from subunit.filters import run_filter_script |
849 | |
850 | try: |
851 | @@ -28,4 +31,6 @@ |
852 | raise |
853 | |
854 | |
855 | -run_filter_script(JUnitXmlResult, __doc__) |
856 | +run_filter_script( |
857 | + lambda output:StreamToExtendedDecorator(JUnitXmlResult(output)), __doc__, |
858 | + protocol_version=2) |
859 | |
860 | === modified file 'filters/subunit2pyunit' |
861 | --- filters/subunit2pyunit 2009-09-30 12:04:18 +0000 |
862 | +++ filters/subunit2pyunit 2013-03-31 05:51:20 +0000 |
863 | @@ -16,11 +16,15 @@ |
864 | |
865 | """Display a subunit stream through python's unittest test runner.""" |
866 | |
867 | +from operator import methodcaller |
868 | from optparse import OptionParser |
869 | import sys |
870 | import unittest |
871 | |
872 | -from subunit import DiscardStream, ProtocolTestCase, TestProtocolServer |
873 | +from testtools import StreamToExtendedDecorator, DecorateTestCaseResult, StreamResultRouter |
874 | + |
875 | +from subunit import ByteStreamToStreamResult |
876 | +from subunit.test_results import CatFiles |
877 | |
878 | parser = OptionParser(description=__doc__) |
879 | parser.add_option("--no-passthrough", action="store_true", |
880 | @@ -29,11 +33,16 @@ |
881 | help="Use bzrlib's test reporter (requires bzrlib)", |
882 | default=False) |
883 | (options, args) = parser.parse_args() |
884 | -if options.no_passthrough: |
885 | - passthrough_stream = DiscardStream() |
886 | -else: |
887 | - passthrough_stream = None |
888 | -test = ProtocolTestCase(sys.stdin, passthrough=passthrough_stream) |
889 | +test = ByteStreamToStreamResult(sys.stdin, non_subunit_name='stdout') |
890 | +def wrap_result(result): |
891 | + result = StreamToExtendedDecorator(result) |
892 | + if not options.no_passthrough: |
893 | + result = StreamResultRouter(result) |
894 | + result.map(CatFiles(sys.stdout), 'test_id', test_id=None) |
895 | + return result |
896 | +test = DecorateTestCaseResult(test, wrap_result, |
897 | + before_run=methodcaller('startTestRun'), |
898 | + after_run=methodcaller('stopTestRun')) |
899 | if options.progress: |
900 | from bzrlib.tests import TextTestRunner |
901 | from bzrlib import ui |
902 | |
903 | === modified file 'python/subunit/__init__.py' |
904 | --- python/subunit/__init__.py 2013-02-07 11:33:11 +0000 |
905 | +++ python/subunit/__init__.py 2013-03-31 05:51:20 +0000 |
906 | @@ -126,7 +126,7 @@ |
907 | except ImportError: |
908 | _UnsupportedOperation = AttributeError |
909 | |
910 | - |
911 | +from extras import safe_hasattr |
912 | from testtools import content, content_type, ExtendedToOriginalDecorator |
913 | from testtools.content import TracebackContent |
914 | from testtools.compat import _b, _u, BytesIO, StringIO |
915 | @@ -143,9 +143,10 @@ |
916 | except ImportError: |
917 | raise ImportError ("testtools.testresult.real does not contain " |
918 | "_StringException, check your version.") |
919 | -from testtools import testresult |
920 | +from testtools import testresult, CopyStreamResult |
921 | |
922 | from subunit import chunked, details, iso8601, test_results |
923 | +from subunit.v2 import ByteStreamToStreamResult, StreamResultToBytes |
924 | |
925 | # same format as sys.version_info: "A tuple containing the five components of |
926 | # the version number: major, minor, micro, releaselevel, and serial. All |
927 | @@ -992,44 +993,51 @@ |
928 | return result |
929 | |
930 | |
931 | -def TAP2SubUnit(tap, subunit): |
932 | +def TAP2SubUnit(tap, output_stream): |
933 | """Filter a TAP pipe into a subunit pipe. |
934 | |
935 | - :param tap: A tap pipe/stream/file object. |
936 | + This should be invoked once per TAP script, as TAP scripts get |
937 | + mapped to a single runnable case with multiple components. |
938 | + |
939 | + :param tap: A tap pipe/stream/file object - should emit unicode strings. |
940 | :param subunit: A pipe/stream/file object to write subunit results to. |
941 | :return: The exit code to exit with. |
942 | """ |
943 | + output = StreamResultToBytes(output_stream) |
944 | + UTF8_TEXT = 'text/plain; charset=UTF8' |
945 | BEFORE_PLAN = 0 |
946 | AFTER_PLAN = 1 |
947 | SKIP_STREAM = 2 |
948 | state = BEFORE_PLAN |
949 | plan_start = 1 |
950 | plan_stop = 0 |
951 | - def _skipped_test(subunit, plan_start): |
952 | - # Some tests were skipped. |
953 | - subunit.write('test test %d\n' % plan_start) |
954 | - subunit.write('error test %d [\n' % plan_start) |
955 | - subunit.write('test missing from TAP output\n') |
956 | - subunit.write(']\n') |
957 | - return plan_start + 1 |
958 | # Test data for the next test to emit |
959 | test_name = None |
960 | log = [] |
961 | result = None |
962 | + def missing_test(plan_start): |
963 | + output.status(test_id='test %d' % plan_start, |
964 | + test_status='fail', runnable=False, |
965 | + mime_type=UTF8_TEXT, eof=True, file_name="tap meta", |
966 | + file_bytes=b"test missing from TAP output") |
967 | def _emit_test(): |
968 | "write out a test" |
969 | if test_name is None: |
970 | return |
971 | - subunit.write("test %s\n" % test_name) |
972 | - if not log: |
973 | - subunit.write("%s %s\n" % (result, test_name)) |
974 | - else: |
975 | - subunit.write("%s %s [\n" % (result, test_name)) |
976 | if log: |
977 | - for line in log: |
978 | - subunit.write("%s\n" % line) |
979 | - subunit.write("]\n") |
980 | + log_bytes = b'\n'.join(log_line.encode('utf8') for log_line in log) |
981 | + mime_type = UTF8_TEXT |
982 | + file_name = 'tap comment' |
983 | + eof = True |
984 | + else: |
985 | + log_bytes = None |
986 | + mime_type = None |
987 | + file_name = None |
988 | + eof = True |
989 | del log[:] |
990 | + output.status(test_id=test_name, test_status=result, |
991 | + file_bytes=log_bytes, mime_type=mime_type, eof=eof, |
992 | + file_name=file_name, runnable=False) |
993 | for line in tap: |
994 | if state == BEFORE_PLAN: |
995 | match = re.match("(\d+)\.\.(\d+)\s*(?:\#\s+(.*))?\n", line) |
996 | @@ -1040,10 +1048,9 @@ |
997 | if plan_start > plan_stop and plan_stop == 0: |
998 | # skipped file |
999 | state = SKIP_STREAM |
1000 | - subunit.write("test file skip\n") |
1001 | - subunit.write("skip file skip [\n") |
1002 | - subunit.write("%s\n" % comment) |
1003 | - subunit.write("]\n") |
1004 | + output.status(test_id='file skip', test_status='skip', |
1005 | + file_bytes=comment.encode('utf8'), eof=True, |
1006 | + file_name='tap comment') |
1007 | continue |
1008 | # not a plan line, or have seen one before |
1009 | match = re.match("(ok|not ok)(?:\s+(\d+)?)?(?:\s+([^#]*[^#\s]+)\s*)?(?:\s+#\s+(TODO|SKIP|skip|todo)(?:\s+(.*))?)?\n", line) |
1010 | @@ -1054,7 +1061,7 @@ |
1011 | if status == 'ok': |
1012 | result = 'success' |
1013 | else: |
1014 | - result = "failure" |
1015 | + result = "fail" |
1016 | if description is None: |
1017 | description = '' |
1018 | else: |
1019 | @@ -1069,7 +1076,8 @@ |
1020 | if number is not None: |
1021 | number = int(number) |
1022 | while plan_start < number: |
1023 | - plan_start = _skipped_test(subunit, plan_start) |
1024 | + missing_test(plan_start) |
1025 | + plan_start += 1 |
1026 | test_name = "test %d%s" % (plan_start, description) |
1027 | plan_start += 1 |
1028 | continue |
1029 | @@ -1082,18 +1090,21 @@ |
1030 | extra = ' %s' % reason |
1031 | _emit_test() |
1032 | test_name = "Bail out!%s" % extra |
1033 | - result = "error" |
1034 | + result = "fail" |
1035 | state = SKIP_STREAM |
1036 | continue |
1037 | match = re.match("\#.*\n", line) |
1038 | if match: |
1039 | log.append(line[:-1]) |
1040 | continue |
1041 | - subunit.write(line) |
1042 | + # Should look at buffering status and binding this to the prior result. |
1043 | + output.status(file_bytes=line.encode('utf8'), file_name='stdout', |
1044 | + mime_type=UTF8_TEXT) |
1045 | _emit_test() |
1046 | while plan_start <= plan_stop: |
1047 | # record missed tests |
1048 | - plan_start = _skipped_test(subunit, plan_start) |
1049 | + missing_test(plan_start) |
1050 | + plan_start += 1 |
1051 | return 0 |
1052 | |
1053 | |
1054 | @@ -1121,24 +1132,21 @@ |
1055 | :return: 0 |
1056 | """ |
1057 | new_tags, gone_tags = tags_to_new_gone(tags) |
1058 | - def write_tags(new_tags, gone_tags): |
1059 | - if new_tags or gone_tags: |
1060 | - filtered.write("tags: " + ' '.join(new_tags)) |
1061 | - if gone_tags: |
1062 | - for tag in gone_tags: |
1063 | - filtered.write("-" + tag) |
1064 | - filtered.write("\n") |
1065 | - write_tags(new_tags, gone_tags) |
1066 | - # TODO: use the protocol parser and thus don't mangle test comments. |
1067 | - for line in original: |
1068 | - if line.startswith("tags:"): |
1069 | - line_tags = line[5:].split() |
1070 | - line_new, line_gone = tags_to_new_gone(line_tags) |
1071 | - line_new = line_new - gone_tags |
1072 | - line_gone = line_gone - new_tags |
1073 | - write_tags(line_new, line_gone) |
1074 | - else: |
1075 | - filtered.write(line) |
1076 | + source = ByteStreamToStreamResult(original, non_subunit_name='stdout') |
1077 | + class Tagger(CopyStreamResult): |
1078 | + def status(self, **kwargs): |
1079 | + tags = kwargs.get('test_tags') |
1080 | + if not tags: |
1081 | + tags = set() |
1082 | + tags.update(new_tags) |
1083 | + tags.difference_update(gone_tags) |
1084 | + if tags: |
1085 | + kwargs['test_tags'] = tags |
1086 | + else: |
1087 | + kwargs['test_tags'] = None |
1088 | + super(Tagger, self).status(**kwargs) |
1089 | + output = Tagger([StreamResultToBytes(filtered)]) |
1090 | + source.run(output) |
1091 | return 0 |
1092 | |
1093 | |
1094 | @@ -1260,7 +1268,8 @@ |
1095 | else: |
1096 | stream = sys.stdout |
1097 | if sys.version_info > (3, 0): |
1098 | - stream = stream.buffer |
1099 | + if safe_hasattr(stream, 'buffer'): |
1100 | + stream = stream.buffer |
1101 | return stream |
1102 | |
1103 | |
1104 | @@ -1291,6 +1300,7 @@ |
1105 | _make_binary_on_windows(fileno) |
1106 | return _unwrap_text(stream) |
1107 | |
1108 | + |
1109 | def _make_binary_on_windows(fileno): |
1110 | """Win32 mangles \r\n to \n and that breaks streams. See bug lp:505078.""" |
1111 | if sys.platform == "win32": |
1112 | |
1113 | === modified file 'python/subunit/filters.py' |
1114 | --- python/subunit/filters.py 2012-03-27 11:17:37 +0000 |
1115 | +++ python/subunit/filters.py 2013-03-31 05:51:20 +0000 |
1116 | @@ -17,7 +17,14 @@ |
1117 | from optparse import OptionParser |
1118 | import sys |
1119 | |
1120 | -from subunit import DiscardStream, ProtocolTestCase |
1121 | +from extras import safe_hasattr |
1122 | +from testtools import CopyStreamResult, StreamResult, StreamResultRouter |
1123 | + |
1124 | +from subunit import ( |
1125 | + DiscardStream, ProtocolTestCase, ByteStreamToStreamResult, |
1126 | + StreamResultToBytes, |
1127 | + ) |
1128 | +from subunit.test_results import CatFiles |
1129 | |
1130 | |
1131 | def make_options(description): |
1132 | @@ -31,33 +38,75 @@ |
1133 | help="Send the output to this path rather than stdout.") |
1134 | parser.add_option( |
1135 | "-f", "--forward", action="store_true", default=False, |
1136 | - help="Forward subunit stream on stdout.") |
1137 | + help="Forward subunit stream on stdout. When set, received " |
1138 | + "non-subunit output will be encapsulated in subunit.") |
1139 | return parser |
1140 | |
1141 | |
1142 | def run_tests_from_stream(input_stream, result, passthrough_stream=None, |
1143 | - forward_stream=None): |
1144 | + forward_stream=None, protocol_version=1, passthrough_subunit=True): |
1145 | """Run tests from a subunit input stream through 'result'. |
1146 | |
1147 | + Non-test events - top level file attachments - are expected to be |
1148 | + dropped by v2 StreamResults at the present time (as all the analysis code |
1149 | + is in ExtendedTestResult API's), so to implement passthrough_stream they |
1150 | + are diverted and copied directly when that is set. |
1151 | + |
1152 | :param input_stream: A stream containing subunit input. |
1153 | :param result: A TestResult that will receive the test events. |
1154 | + NB: This should be an ExtendedTestResult for v1 and a StreamResult for |
1155 | + v2. |
1156 | :param passthrough_stream: All non-subunit input received will be |
1157 | sent to this stream. If not provided, uses the ``TestProtocolServer`` |
1158 | default, which is ``sys.stdout``. |
1159 | :param forward_stream: All subunit input received will be forwarded |
1160 | - to this stream. If not provided, uses the ``TestProtocolServer`` |
1161 | - default, which is to not forward any input. |
1162 | + to this stream. If not provided, uses the ``TestProtocolServer`` |
1163 | + default, which is to not forward any input. Do not set this when |
1164 | + transforming the stream - items would be double-reported. |
1165 | + :param protocol_version: What version of the subunit protocol to expect. |
1166 | + :param passthrough_subunit: If True, passthrough should be as subunit |
1167 | + otherwise unwrap it. Only has effect when forward_stream is None. |
1168 | + (when forwarding as subunit non-subunit input is always turned into |
1169 | + subunit) |
1170 | """ |
1171 | - test = ProtocolTestCase( |
1172 | - input_stream, passthrough=passthrough_stream, |
1173 | - forward=forward_stream) |
1174 | + if 1==protocol_version: |
1175 | + test = ProtocolTestCase( |
1176 | + input_stream, passthrough=passthrough_stream, |
1177 | + forward=forward_stream) |
1178 | + elif 2==protocol_version: |
1179 | + # In all cases we encapsulate unknown inputs. |
1180 | + if forward_stream is not None: |
1181 | + # Send events to forward_stream as subunit. |
1182 | + forward_result = StreamResultToBytes(forward_stream) |
1183 | + # If we're passing non-subunit through, copy: |
1184 | + if passthrough_stream is None: |
1185 | + # Not passing non-test events - split them off to nothing. |
1186 | + router = StreamResultRouter(forward_result) |
1187 | + router.map(StreamResult(), 'test_id', test_id=None) |
1188 | + result = CopyStreamResult([router, result]) |
1189 | + else: |
1190 | + # otherwise, copy all events to forward_result |
1191 | + result = CopyStreamResult([forward_result, result]) |
1192 | + elif passthrough_stream is not None: |
1193 | + if not passthrough_subunit: |
1194 | + # Route non-test events to passthrough_stream, unwrapping them for |
1195 | + # display. |
1196 | + passthrough_result = CatFiles(passthrough_stream) |
1197 | + else: |
1198 | + passthrough_result = StreamResultToBytes(passthrough_stream) |
1199 | + result = StreamResultRouter(result) |
1200 | + result.map(passthrough_result, 'test_id', test_id=None) |
1201 | + test = ByteStreamToStreamResult(input_stream, |
1202 | + non_subunit_name='stdout') |
1203 | + else: |
1204 | + raise Exception("Unknown protocol version.") |
1205 | result.startTestRun() |
1206 | test.run(result) |
1207 | result.stopTestRun() |
1208 | |
1209 | |
1210 | def filter_by_result(result_factory, output_path, passthrough, forward, |
1211 | - input_stream=sys.stdin): |
1212 | + input_stream=sys.stdin, protocol_version=1): |
1213 | """Filter an input stream using a test result. |
1214 | |
1215 | :param result_factory: A callable that when passed an output stream |
1216 | @@ -71,17 +120,23 @@ |
1217 | ``sys.stdout`` as well as to the ``TestResult``. |
1218 | :param input_stream: The source of subunit input. Defaults to |
1219 | ``sys.stdin``. |
1220 | - :return: A test result with the resultts of the run. |
1221 | + :param protocol_version: The subunit protocol version to expect. |
1222 | + :return: A test result with the results of the run. |
1223 | """ |
1224 | if passthrough: |
1225 | passthrough_stream = sys.stdout |
1226 | else: |
1227 | - passthrough_stream = DiscardStream() |
1228 | + if 1==protocol_version: |
1229 | + passthrough_stream = DiscardStream() |
1230 | + else: |
1231 | + passthrough_stream = None |
1232 | |
1233 | if forward: |
1234 | forward_stream = sys.stdout |
1235 | - else: |
1236 | + elif 1==protocol_version: |
1237 | forward_stream = DiscardStream() |
1238 | + else: |
1239 | + forward_stream = None |
1240 | |
1241 | if output_path is None: |
1242 | output_to = sys.stdout |
1243 | @@ -91,14 +146,16 @@ |
1244 | try: |
1245 | result = result_factory(output_to) |
1246 | run_tests_from_stream( |
1247 | - input_stream, result, passthrough_stream, forward_stream) |
1248 | + input_stream, result, passthrough_stream, forward_stream, |
1249 | + protocol_version=protocol_version) |
1250 | finally: |
1251 | if output_path: |
1252 | output_to.close() |
1253 | return result |
1254 | |
1255 | |
1256 | -def run_filter_script(result_factory, description, post_run_hook=None): |
1257 | +def run_filter_script(result_factory, description, post_run_hook=None, |
1258 | + protocol_version=1): |
1259 | """Main function for simple subunit filter scripts. |
1260 | |
1261 | Many subunit filter scripts take a stream of subunit input and use a |
1262 | @@ -111,14 +168,17 @@ |
1263 | :param result_factory: A callable that takes an output stream and returns |
1264 | a test result that outputs to that stream. |
1265 | :param description: A description of the filter script. |
1266 | + :param protocol_version: What protocol version to consume/emit. |
1267 | """ |
1268 | parser = make_options(description) |
1269 | (options, args) = parser.parse_args() |
1270 | result = filter_by_result( |
1271 | result_factory, options.output_to, not options.no_passthrough, |
1272 | - options.forward) |
1273 | + options.forward, protocol_version=protocol_version) |
1274 | if post_run_hook: |
1275 | post_run_hook(result) |
1276 | + if not safe_hasattr(result, 'wasSuccessful'): |
1277 | + result = result.decorated |
1278 | if result.wasSuccessful(): |
1279 | sys.exit(0) |
1280 | else: |
1281 | |
1282 | === modified file 'python/subunit/run.py' |
1283 | --- python/subunit/run.py 2012-12-17 07:24:28 +0000 |
1284 | +++ python/subunit/run.py 2013-03-31 05:51:20 +0000 |
1285 | @@ -20,9 +20,14 @@ |
1286 | $ python -m subunit.run mylib.tests.test_suite |
1287 | """ |
1288 | |
1289 | +import io |
1290 | +import os |
1291 | import sys |
1292 | |
1293 | -from subunit import TestProtocolClient, get_default_formatter |
1294 | +from testtools import ExtendedToStreamDecorator |
1295 | +from testtools.testsuite import iterate_tests |
1296 | + |
1297 | +from subunit import StreamResultToBytes, get_default_formatter |
1298 | from subunit.test_results import AutoTimingTestResultDecorator |
1299 | from testtools.run import ( |
1300 | BUFFEROUTPUT, |
1301 | @@ -46,11 +51,34 @@ |
1302 | |
1303 | def run(self, test): |
1304 | "Run the given test case or test suite." |
1305 | - result = TestProtocolClient(self.stream) |
1306 | + result = self._list(test) |
1307 | + result = ExtendedToStreamDecorator(result) |
1308 | result = AutoTimingTestResultDecorator(result) |
1309 | if self.failfast is not None: |
1310 | result.failfast = self.failfast |
1311 | - test(result) |
1312 | + result.startTestRun() |
1313 | + try: |
1314 | + test(result) |
1315 | + finally: |
1316 | + result.stopTestRun() |
1317 | + return result |
1318 | + |
1319 | + def list(self, test): |
1320 | + "List the test." |
1321 | + self._list(test) |
1322 | + |
1323 | + def _list(self, test): |
1324 | + try: |
1325 | + fileno = self.stream.fileno() |
1326 | + except: |
1327 | + fileno = None |
1328 | + if fileno is not None: |
1329 | + stream = os.fdopen(fileno, 'wb', 0) |
1330 | + else: |
1331 | + stream = self.stream |
1332 | + result = StreamResultToBytes(stream) |
1333 | + for case in iterate_tests(test): |
1334 | + result.status(test_id=case.id(), test_status='exists') |
1335 | return result |
1336 | |
1337 | |
1338 | @@ -78,7 +106,15 @@ |
1339 | |
1340 | |
1341 | if __name__ == '__main__': |
1342 | + # Disable the default buffering, for Python 2.x where pdb doesn't do it |
1343 | + # on non-ttys. |
1344 | stream = get_default_formatter() |
1345 | runner = SubunitTestRunner |
1346 | + # Patch stdout to be unbuffered, so that pdb works well on 2.6/2.7. |
1347 | + binstdout = io.open(sys.stdout.fileno(), 'wb', 0) |
1348 | + if sys.version_info[0] > 2: |
1349 | + sys.stdout = io.TextIOWrapper(binstdout, encoding=sys.stdout.encoding) |
1350 | + else: |
1351 | + sys.stdout = binstdout |
1352 | SubunitTestProgram(module=None, argv=sys.argv, testRunner=runner, |
1353 | stdout=sys.stdout) |
1354 | |
1355 | === modified file 'python/subunit/test_results.py' |
1356 | --- python/subunit/test_results.py 2012-12-17 07:24:28 +0000 |
1357 | +++ python/subunit/test_results.py 2013-03-31 05:51:20 +0000 |
1358 | @@ -25,8 +25,10 @@ |
1359 | text_content, |
1360 | TracebackContent, |
1361 | ) |
1362 | +from testtools import StreamResult |
1363 | |
1364 | from subunit import iso8601 |
1365 | +import subunit |
1366 | |
1367 | |
1368 | # NOT a TestResult, because we are implementing the interface, not inheriting |
1369 | @@ -525,16 +527,24 @@ |
1370 | |
1371 | |
1372 | class TestIdPrintingResult(testtools.TestResult): |
1373 | - |
1374 | - def __init__(self, stream, show_times=False): |
1375 | + """Print test ids to a stream. |
1376 | + |
1377 | + Implements both TestResult and StreamResult, for compatibility. |
1378 | + """ |
1379 | + |
1380 | + def __init__(self, stream, show_times=False, show_exists=False): |
1381 | """Create a FilterResult object outputting to stream.""" |
1382 | super(TestIdPrintingResult, self).__init__() |
1383 | self._stream = stream |
1384 | + self.show_exists = show_exists |
1385 | + self.show_times = show_times |
1386 | + |
1387 | + def startTestRun(self): |
1388 | self.failed_tests = 0 |
1389 | self.__time = None |
1390 | - self.show_times = show_times |
1391 | self._test = None |
1392 | self._test_duration = 0 |
1393 | + self._active_tests = {} |
1394 | |
1395 | def addError(self, test, err): |
1396 | self.failed_tests += 1 |
1397 | @@ -557,21 +567,44 @@ |
1398 | def addExpectedFailure(self, test, err=None, details=None): |
1399 | self._test = test |
1400 | |
1401 | - def reportTest(self, test, duration): |
1402 | + def reportTest(self, test_id, duration): |
1403 | if self.show_times: |
1404 | seconds = duration.seconds |
1405 | seconds += duration.days * 3600 * 24 |
1406 | seconds += duration.microseconds / 1000000.0 |
1407 | - self._stream.write(test.id() + ' %0.3f\n' % seconds) |
1408 | + self._stream.write(test_id + ' %0.3f\n' % seconds) |
1409 | else: |
1410 | - self._stream.write(test.id() + '\n') |
1411 | + self._stream.write(test_id + '\n') |
1412 | |
1413 | def startTest(self, test): |
1414 | self._start_time = self._time() |
1415 | |
1416 | + def status(self, test_id=None, test_status=None, test_tags=None, |
1417 | + runnable=True, file_name=None, file_bytes=None, eof=False, |
1418 | + mime_type=None, route_code=None, timestamp=None): |
1419 | + if not test_id: |
1420 | + return |
1421 | + if timestamp is not None: |
1422 | + self.time(timestamp) |
1423 | + if test_status=='exists': |
1424 | + if self.show_exists: |
1425 | + self.reportTest(test_id, 0) |
1426 | + elif test_status in ('inprogress', None): |
1427 | + self._active_tests[test_id] = self._time() |
1428 | + else: |
1429 | + self._end_test(test_id) |
1430 | + |
1431 | + def _end_test(self, test_id): |
1432 | + test_start = self._active_tests.pop(test_id, None) |
1433 | + if not test_start: |
1434 | + test_duration = 0 |
1435 | + else: |
1436 | + test_duration = self._time() - test_start |
1437 | + self.reportTest(test_id, test_duration) |
1438 | + |
1439 | def stopTest(self, test): |
1440 | test_duration = self._time() - self._start_time |
1441 | - self.reportTest(self._test, test_duration) |
1442 | + self.reportTest(self._test.id(), test_duration) |
1443 | |
1444 | def time(self, time): |
1445 | self.__time = time |
1446 | @@ -583,6 +616,10 @@ |
1447 | "Tells whether or not this result was a success" |
1448 | return self.failed_tests == 0 |
1449 | |
1450 | + def stopTestRun(self): |
1451 | + for test_id in list(self._active_tests.keys()): |
1452 | + self._end_test(test_id) |
1453 | + |
1454 | |
1455 | class TestByTestResult(testtools.TestResult): |
1456 | """Call something every time a test completes.""" |
1457 | @@ -676,3 +713,17 @@ |
1458 | def startTestRun(self): |
1459 | super(CsvResult, self).startTestRun() |
1460 | self._write_row(['test', 'status', 'start_time', 'stop_time']) |
1461 | + |
1462 | + |
1463 | +class CatFiles(StreamResult): |
1464 | + """Cat file attachments received to a stream.""" |
1465 | + |
1466 | + def __init__(self, byte_stream): |
1467 | + self.stream = subunit.make_stream_binary(byte_stream) |
1468 | + |
1469 | + def status(self, test_id=None, test_status=None, test_tags=None, |
1470 | + runnable=True, file_name=None, file_bytes=None, eof=False, |
1471 | + mime_type=None, route_code=None, timestamp=None): |
1472 | + if file_name is not None: |
1473 | + self.stream.write(file_bytes) |
1474 | + self.stream.flush() |
1475 | |
1476 | === modified file 'python/subunit/tests/__init__.py' |
1477 | --- python/subunit/tests/__init__.py 2011-11-01 15:59:58 +0000 |
1478 | +++ python/subunit/tests/__init__.py 2013-03-31 05:51:20 +0000 |
1479 | @@ -25,6 +25,7 @@ |
1480 | test_subunit_tags, |
1481 | test_tap2subunit, |
1482 | test_test_protocol, |
1483 | + test_test_protocol2, |
1484 | test_test_results, |
1485 | ) |
1486 | |
1487 | @@ -35,6 +36,7 @@ |
1488 | result.addTest(test_progress_model.test_suite()) |
1489 | result.addTest(test_test_results.test_suite()) |
1490 | result.addTest(test_test_protocol.test_suite()) |
1491 | + result.addTest(test_test_protocol2.test_suite()) |
1492 | result.addTest(test_tap2subunit.test_suite()) |
1493 | result.addTest(test_subunit_filter.test_suite()) |
1494 | result.addTest(test_subunit_tags.test_suite()) |
1495 | |
1496 | === modified file 'python/subunit/tests/test_run.py' |
1497 | --- python/subunit/tests/test_run.py 2012-05-07 19:36:05 +0000 |
1498 | +++ python/subunit/tests/test_run.py 2013-03-31 05:51:20 +0000 |
1499 | @@ -18,6 +18,7 @@ |
1500 | import unittest |
1501 | |
1502 | from testtools import PlaceHolder |
1503 | +from testtools.testresult.doubles import StreamResult |
1504 | |
1505 | import subunit |
1506 | from subunit.run import SubunitTestRunner |
1507 | @@ -29,16 +30,6 @@ |
1508 | return result |
1509 | |
1510 | |
1511 | -class TimeCollectingTestResult(unittest.TestResult): |
1512 | - |
1513 | - def __init__(self, *args, **kwargs): |
1514 | - super(TimeCollectingTestResult, self).__init__(*args, **kwargs) |
1515 | - self.time_called = [] |
1516 | - |
1517 | - def time(self, a_time): |
1518 | - self.time_called.append(a_time) |
1519 | - |
1520 | - |
1521 | class TestSubunitTestRunner(unittest.TestCase): |
1522 | |
1523 | def test_includes_timing_output(self): |
1524 | @@ -46,7 +37,24 @@ |
1525 | runner = SubunitTestRunner(stream=io) |
1526 | test = PlaceHolder('name') |
1527 | runner.run(test) |
1528 | - client = TimeCollectingTestResult() |
1529 | - io.seek(0) |
1530 | - subunit.TestProtocolServer(client).readFrom(io) |
1531 | - self.assertTrue(len(client.time_called) > 0) |
1532 | + io.seek(0) |
1533 | + eventstream = StreamResult() |
1534 | + subunit.ByteStreamToStreamResult(io).run(eventstream) |
1535 | + timestamps = [event[-1] for event in eventstream._events |
1536 | + if event is not None] |
1537 | + self.assertNotEqual([], timestamps) |
1538 | + |
1539 | + def test_enumerates_tests_before_run(self): |
1540 | + io = BytesIO() |
1541 | + runner = SubunitTestRunner(stream=io) |
1542 | + test1 = PlaceHolder('name1') |
1543 | + test2 = PlaceHolder('name2') |
1544 | + case = unittest.TestSuite([test1, test2]) |
1545 | + runner.run(case) |
1546 | + io.seek(0) |
1547 | + eventstream = StreamResult() |
1548 | + subunit.ByteStreamToStreamResult(io).run(eventstream) |
1549 | + self.assertEqual([ |
1550 | + ('status', 'name1', 'exists'), |
1551 | + ('status', 'name2', 'exists'), |
1552 | + ], [event[:3] for event in eventstream._events[:2]]) |
1553 | |
1554 | === modified file 'python/subunit/tests/test_subunit_filter.py' |
1555 | --- python/subunit/tests/test_subunit_filter.py 2012-05-07 22:53:53 +0000 |
1556 | +++ python/subunit/tests/test_subunit_filter.py 2013-03-31 05:51:20 +0000 |
1557 | @@ -25,10 +25,11 @@ |
1558 | |
1559 | from testtools import TestCase |
1560 | from testtools.compat import _b, BytesIO |
1561 | -from testtools.testresult.doubles import ExtendedTestResult |
1562 | +from testtools.testresult.doubles import ExtendedTestResult, StreamResult |
1563 | |
1564 | import subunit |
1565 | from subunit.test_results import make_tag_filter, TestResultFilter |
1566 | +from subunit import ByteStreamToStreamResult, StreamResultToBytes |
1567 | |
1568 | |
1569 | class TestTestResultFilter(TestCase): |
1570 | @@ -286,23 +287,6 @@ |
1571 | |
1572 | class TestFilterCommand(TestCase): |
1573 | |
1574 | - example_subunit_stream = _b("""\ |
1575 | -tags: global |
1576 | -test passed |
1577 | -success passed |
1578 | -test failed |
1579 | -tags: local |
1580 | -failure failed |
1581 | -test error |
1582 | -error error [ |
1583 | -error details |
1584 | -] |
1585 | -test skipped |
1586 | -skip skipped |
1587 | -test todo |
1588 | -xfail todo |
1589 | -""") |
1590 | - |
1591 | def run_command(self, args, stream): |
1592 | root = os.path.dirname( |
1593 | os.path.dirname(os.path.dirname(os.path.dirname(__file__)))) |
1594 | @@ -316,52 +300,50 @@ |
1595 | raise RuntimeError("%s failed: %s" % (command, err)) |
1596 | return out |
1597 | |
1598 | - def to_events(self, stream): |
1599 | - test = subunit.ProtocolTestCase(BytesIO(stream)) |
1600 | - result = ExtendedTestResult() |
1601 | - test.run(result) |
1602 | - return result._events |
1603 | - |
1604 | def test_default(self): |
1605 | - output = self.run_command([], _b( |
1606 | - "test: foo\n" |
1607 | - "skip: foo\n" |
1608 | - )) |
1609 | - events = self.to_events(output) |
1610 | - foo = subunit.RemotedTestCase('foo') |
1611 | - self.assertEqual( |
1612 | - [('startTest', foo), |
1613 | - ('addSkip', foo, {}), |
1614 | - ('stopTest', foo)], |
1615 | - events) |
1616 | + byte_stream = BytesIO() |
1617 | + stream = StreamResultToBytes(byte_stream) |
1618 | + stream.status(test_id="foo", test_status="inprogress") |
1619 | + stream.status(test_id="foo", test_status="skip") |
1620 | + output = self.run_command([], byte_stream.getvalue()) |
1621 | + events = StreamResult() |
1622 | + ByteStreamToStreamResult(BytesIO(output)).run(events) |
1623 | + ids = set(event[1] for event in events._events) |
1624 | + self.assertEqual([ |
1625 | + ('status', 'foo', 'inprogress'), |
1626 | + ('status', 'foo', 'skip'), |
1627 | + ], [event[:3] for event in events._events]) |
1628 | |
1629 | def test_tags(self): |
1630 | - output = self.run_command(['-s', '--with-tag', 'a'], _b( |
1631 | - "tags: a\n" |
1632 | - "test: foo\n" |
1633 | - "success: foo\n" |
1634 | - "tags: -a\n" |
1635 | - "test: bar\n" |
1636 | - "success: bar\n" |
1637 | - "test: baz\n" |
1638 | - "tags: a\n" |
1639 | - "success: baz\n" |
1640 | - )) |
1641 | - events = self.to_events(output) |
1642 | - foo = subunit.RemotedTestCase('foo') |
1643 | - baz = subunit.RemotedTestCase('baz') |
1644 | - self.assertEqual( |
1645 | - [('tags', set(['a']), set()), |
1646 | - ('startTest', foo), |
1647 | - ('addSuccess', foo), |
1648 | - ('stopTest', foo), |
1649 | - ('tags', set(), set(['a'])), |
1650 | - ('startTest', baz), |
1651 | - ('tags', set(['a']), set()), |
1652 | - ('addSuccess', baz), |
1653 | - ('stopTest', baz), |
1654 | - ], |
1655 | - events) |
1656 | + byte_stream = BytesIO() |
1657 | + stream = StreamResultToBytes(byte_stream) |
1658 | + stream.status( |
1659 | + test_id="foo", test_status="inprogress", test_tags=set(["a"])) |
1660 | + stream.status( |
1661 | + test_id="foo", test_status="success", test_tags=set(["a"])) |
1662 | + stream.status(test_id="bar", test_status="inprogress") |
1663 | + stream.status(test_id="bar", test_status="inprogress") |
1664 | + stream.status( |
1665 | + test_id="baz", test_status="inprogress", test_tags=set(["a"])) |
1666 | + stream.status( |
1667 | + test_id="baz", test_status="success", test_tags=set(["a"])) |
1668 | + output = self.run_command( |
1669 | + ['-s', '--with-tag', 'a'], byte_stream.getvalue()) |
1670 | + events = StreamResult() |
1671 | + ByteStreamToStreamResult(BytesIO(output)).run(events) |
1672 | + ids = set(event[1] for event in events._events) |
1673 | + self.assertEqual(set(['foo', 'baz']), ids) |
1674 | + |
1675 | + def test_no_passthrough(self): |
1676 | + output = self.run_command(['--no-passthrough'], b'hi thar') |
1677 | + self.assertEqual(b'', output) |
1678 | + |
1679 | + def test_passthrough(self): |
1680 | + output = self.run_command([], b'hi thar') |
1681 | + byte_stream = BytesIO() |
1682 | + stream = StreamResultToBytes(byte_stream) |
1683 | + stream.status(file_name="stdout", file_bytes=b'hi thar') |
1684 | + self.assertEqual(byte_stream.getvalue(), output) |
1685 | |
1686 | |
1687 | def test_suite(): |
1688 | |
1689 | === modified file 'python/subunit/tests/test_subunit_tags.py' |
1690 | --- python/subunit/tests/test_subunit_tags.py 2011-04-24 21:40:52 +0000 |
1691 | +++ python/subunit/tests/test_subunit_tags.py 2013-03-31 05:51:20 +0000 |
1692 | @@ -16,10 +16,9 @@ |
1693 | |
1694 | """Tests for subunit.tag_stream.""" |
1695 | |
1696 | +from io import BytesIO |
1697 | import unittest |
1698 | |
1699 | -from testtools.compat import StringIO |
1700 | - |
1701 | import subunit |
1702 | import subunit.test_results |
1703 | |
1704 | @@ -27,40 +26,42 @@ |
1705 | class TestSubUnitTags(unittest.TestCase): |
1706 | |
1707 | def setUp(self): |
1708 | - self.original = StringIO() |
1709 | - self.filtered = StringIO() |
1710 | + self.original = BytesIO() |
1711 | + self.filtered = BytesIO() |
1712 | |
1713 | def test_add_tag(self): |
1714 | - self.original.write("tags: foo\n") |
1715 | - self.original.write("test: test\n") |
1716 | - self.original.write("tags: bar -quux\n") |
1717 | - self.original.write("success: test\n") |
1718 | + reference = BytesIO() |
1719 | + stream = subunit.StreamResultToBytes(reference) |
1720 | + stream.status( |
1721 | + test_id='test', test_status='inprogress', test_tags=set(['quux', 'foo'])) |
1722 | + stream.status( |
1723 | + test_id='test', test_status='success', test_tags=set(['bar', 'quux', 'foo'])) |
1724 | + stream = subunit.StreamResultToBytes(self.original) |
1725 | + stream.status( |
1726 | + test_id='test', test_status='inprogress', test_tags=set(['foo'])) |
1727 | + stream.status( |
1728 | + test_id='test', test_status='success', test_tags=set(['foo', 'bar'])) |
1729 | self.original.seek(0) |
1730 | - result = subunit.tag_stream(self.original, self.filtered, ["quux"]) |
1731 | - self.assertEqual([ |
1732 | - "tags: quux", |
1733 | - "tags: foo", |
1734 | - "test: test", |
1735 | - "tags: bar", |
1736 | - "success: test", |
1737 | - ], |
1738 | - self.filtered.getvalue().splitlines()) |
1739 | + self.assertEqual( |
1740 | + 0, subunit.tag_stream(self.original, self.filtered, ["quux"])) |
1741 | + self.assertEqual(reference.getvalue(), self.filtered.getvalue()) |
1742 | |
1743 | def test_remove_tag(self): |
1744 | - self.original.write("tags: foo\n") |
1745 | - self.original.write("test: test\n") |
1746 | - self.original.write("tags: bar -quux\n") |
1747 | - self.original.write("success: test\n") |
1748 | + reference = BytesIO() |
1749 | + stream = subunit.StreamResultToBytes(reference) |
1750 | + stream.status( |
1751 | + test_id='test', test_status='inprogress', test_tags=set(['foo'])) |
1752 | + stream.status( |
1753 | + test_id='test', test_status='success', test_tags=set(['foo'])) |
1754 | + stream = subunit.StreamResultToBytes(self.original) |
1755 | + stream.status( |
1756 | + test_id='test', test_status='inprogress', test_tags=set(['foo'])) |
1757 | + stream.status( |
1758 | + test_id='test', test_status='success', test_tags=set(['foo', 'bar'])) |
1759 | self.original.seek(0) |
1760 | - result = subunit.tag_stream(self.original, self.filtered, ["-bar"]) |
1761 | - self.assertEqual([ |
1762 | - "tags: -bar", |
1763 | - "tags: foo", |
1764 | - "test: test", |
1765 | - "tags: -quux", |
1766 | - "success: test", |
1767 | - ], |
1768 | - self.filtered.getvalue().splitlines()) |
1769 | + self.assertEqual( |
1770 | + 0, subunit.tag_stream(self.original, self.filtered, ["-bar"])) |
1771 | + self.assertEqual(reference.getvalue(), self.filtered.getvalue()) |
1772 | |
1773 | |
1774 | def test_suite(): |
1775 | |
1776 | === modified file 'python/subunit/tests/test_tap2subunit.py' |
1777 | --- python/subunit/tests/test_tap2subunit.py 2011-04-24 21:40:52 +0000 |
1778 | +++ python/subunit/tests/test_tap2subunit.py 2013-03-31 05:51:20 +0000 |
1779 | @@ -16,14 +16,19 @@ |
1780 | |
1781 | """Tests for TAP2SubUnit.""" |
1782 | |
1783 | +from io import BytesIO, StringIO |
1784 | import unittest |
1785 | |
1786 | -from testtools.compat import StringIO |
1787 | +from testtools import TestCase |
1788 | +from testtools.compat import _u |
1789 | +from testtools.testresult.doubles import StreamResult |
1790 | |
1791 | import subunit |
1792 | |
1793 | - |
1794 | -class TestTAP2SubUnit(unittest.TestCase): |
1795 | +UTF8_TEXT = 'text/plain; charset=UTF8' |
1796 | + |
1797 | + |
1798 | +class TestTAP2SubUnit(TestCase): |
1799 | """Tests for TAP2SubUnit. |
1800 | |
1801 | These tests test TAP string data in, and subunit string data out. |
1802 | @@ -34,24 +39,21 @@ |
1803 | """ |
1804 | |
1805 | def setUp(self): |
1806 | + super(TestTAP2SubUnit, self).setUp() |
1807 | self.tap = StringIO() |
1808 | - self.subunit = StringIO() |
1809 | + self.subunit = BytesIO() |
1810 | |
1811 | def test_skip_entire_file(self): |
1812 | # A file |
1813 | # 1..- # Skipped: comment |
1814 | # results in a single skipped test. |
1815 | - self.tap.write("1..0 # Skipped: entire file skipped\n") |
1816 | + self.tap.write(_u("1..0 # Skipped: entire file skipped\n")) |
1817 | self.tap.seek(0) |
1818 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
1819 | self.assertEqual(0, result) |
1820 | - self.assertEqual([ |
1821 | - "test file skip", |
1822 | - "skip file skip [", |
1823 | - "Skipped: entire file skipped", |
1824 | - "]", |
1825 | - ], |
1826 | - self.subunit.getvalue().splitlines()) |
1827 | + self.check_events([('status', 'file skip', 'skip', None, True, |
1828 | + 'tap comment', b'Skipped: entire file skipped', True, None, None, |
1829 | + None)]) |
1830 | |
1831 | def test_ok_test_pass(self): |
1832 | # A file |
1833 | @@ -59,164 +61,128 @@ |
1834 | # results in a passed test with name 'test 1' (a synthetic name as tap |
1835 | # does not require named fixtures - it is the first test in the tap |
1836 | # stream). |
1837 | - self.tap.write("ok\n") |
1838 | + self.tap.write(_u("ok\n")) |
1839 | self.tap.seek(0) |
1840 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
1841 | self.assertEqual(0, result) |
1842 | - self.assertEqual([ |
1843 | - "test test 1", |
1844 | - "success test 1", |
1845 | - ], |
1846 | - self.subunit.getvalue().splitlines()) |
1847 | + self.check_events([('status', 'test 1', 'success', None, False, None, |
1848 | + None, True, None, None, None)]) |
1849 | |
1850 | def test_ok_test_number_pass(self): |
1851 | # A file |
1852 | # ok 1 |
1853 | # results in a passed test with name 'test 1' |
1854 | - self.tap.write("ok 1\n") |
1855 | + self.tap.write(_u("ok 1\n")) |
1856 | self.tap.seek(0) |
1857 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
1858 | self.assertEqual(0, result) |
1859 | - self.assertEqual([ |
1860 | - "test test 1", |
1861 | - "success test 1", |
1862 | - ], |
1863 | - self.subunit.getvalue().splitlines()) |
1864 | + self.check_events([('status', 'test 1', 'success', None, False, None, |
1865 | + None, True, None, None, None)]) |
1866 | |
1867 | def test_ok_test_number_description_pass(self): |
1868 | # A file |
1869 | # ok 1 - There is a description |
1870 | # results in a passed test with name 'test 1 - There is a description' |
1871 | - self.tap.write("ok 1 - There is a description\n") |
1872 | + self.tap.write(_u("ok 1 - There is a description\n")) |
1873 | self.tap.seek(0) |
1874 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
1875 | self.assertEqual(0, result) |
1876 | - self.assertEqual([ |
1877 | - "test test 1 - There is a description", |
1878 | - "success test 1 - There is a description", |
1879 | - ], |
1880 | - self.subunit.getvalue().splitlines()) |
1881 | + self.check_events([('status', 'test 1 - There is a description', |
1882 | + 'success', None, False, None, None, True, None, None, None)]) |
1883 | |
1884 | def test_ok_test_description_pass(self): |
1885 | # A file |
1886 | # ok There is a description |
1887 | # results in a passed test with name 'test 1 There is a description' |
1888 | - self.tap.write("ok There is a description\n") |
1889 | + self.tap.write(_u("ok There is a description\n")) |
1890 | self.tap.seek(0) |
1891 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
1892 | self.assertEqual(0, result) |
1893 | - self.assertEqual([ |
1894 | - "test test 1 There is a description", |
1895 | - "success test 1 There is a description", |
1896 | - ], |
1897 | - self.subunit.getvalue().splitlines()) |
1898 | + self.check_events([('status', 'test 1 There is a description', |
1899 | + 'success', None, False, None, None, True, None, None, None)]) |
1900 | |
1901 | def test_ok_SKIP_skip(self): |
1902 | # A file |
1903 | # ok # SKIP |
1904 | # results in a skkip test with name 'test 1' |
1905 | - self.tap.write("ok # SKIP\n") |
1906 | + self.tap.write(_u("ok # SKIP\n")) |
1907 | self.tap.seek(0) |
1908 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
1909 | self.assertEqual(0, result) |
1910 | - self.assertEqual([ |
1911 | - "test test 1", |
1912 | - "skip test 1", |
1913 | - ], |
1914 | - self.subunit.getvalue().splitlines()) |
1915 | + self.check_events([('status', 'test 1', 'skip', None, False, None, |
1916 | + None, True, None, None, None)]) |
1917 | |
1918 | def test_ok_skip_number_comment_lowercase(self): |
1919 | - self.tap.write("ok 1 # skip no samba environment available, skipping compilation\n") |
1920 | + self.tap.write(_u("ok 1 # skip no samba environment available, skipping compilation\n")) |
1921 | self.tap.seek(0) |
1922 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
1923 | self.assertEqual(0, result) |
1924 | - self.assertEqual([ |
1925 | - "test test 1", |
1926 | - "skip test 1 [", |
1927 | - "no samba environment available, skipping compilation", |
1928 | - "]" |
1929 | - ], |
1930 | - self.subunit.getvalue().splitlines()) |
1931 | + self.check_events([('status', 'test 1', 'skip', None, False, 'tap comment', |
1932 | + b'no samba environment available, skipping compilation', True, |
1933 | + 'text/plain; charset=UTF8', None, None)]) |
1934 | |
1935 | def test_ok_number_description_SKIP_skip_comment(self): |
1936 | # A file |
1937 | # ok 1 foo # SKIP Not done yet |
1938 | # results in a skip test with name 'test 1 foo' and a log of |
1939 | # Not done yet |
1940 | - self.tap.write("ok 1 foo # SKIP Not done yet\n") |
1941 | + self.tap.write(_u("ok 1 foo # SKIP Not done yet\n")) |
1942 | self.tap.seek(0) |
1943 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
1944 | self.assertEqual(0, result) |
1945 | - self.assertEqual([ |
1946 | - "test test 1 foo", |
1947 | - "skip test 1 foo [", |
1948 | - "Not done yet", |
1949 | - "]", |
1950 | - ], |
1951 | - self.subunit.getvalue().splitlines()) |
1952 | + self.check_events([('status', 'test 1 foo', 'skip', None, False, |
1953 | + 'tap comment', b'Not done yet', True, 'text/plain; charset=UTF8', |
1954 | + None, None)]) |
1955 | |
1956 | def test_ok_SKIP_skip_comment(self): |
1957 | # A file |
1958 | # ok # SKIP Not done yet |
1959 | # results in a skip test with name 'test 1' and a log of Not done yet |
1960 | - self.tap.write("ok # SKIP Not done yet\n") |
1961 | + self.tap.write(_u("ok # SKIP Not done yet\n")) |
1962 | self.tap.seek(0) |
1963 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
1964 | self.assertEqual(0, result) |
1965 | - self.assertEqual([ |
1966 | - "test test 1", |
1967 | - "skip test 1 [", |
1968 | - "Not done yet", |
1969 | - "]", |
1970 | - ], |
1971 | - self.subunit.getvalue().splitlines()) |
1972 | + self.check_events([('status', 'test 1', 'skip', None, False, |
1973 | + 'tap comment', b'Not done yet', True, 'text/plain; charset=UTF8', |
1974 | + None, None)]) |
1975 | |
1976 | def test_ok_TODO_xfail(self): |
1977 | # A file |
1978 | # ok # TODO |
1979 | # results in a xfail test with name 'test 1' |
1980 | - self.tap.write("ok # TODO\n") |
1981 | + self.tap.write(_u("ok # TODO\n")) |
1982 | self.tap.seek(0) |
1983 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
1984 | self.assertEqual(0, result) |
1985 | - self.assertEqual([ |
1986 | - "test test 1", |
1987 | - "xfail test 1", |
1988 | - ], |
1989 | - self.subunit.getvalue().splitlines()) |
1990 | + self.check_events([('status', 'test 1', 'xfail', None, False, None, |
1991 | + None, True, None, None, None)]) |
1992 | |
1993 | def test_ok_TODO_xfail_comment(self): |
1994 | # A file |
1995 | # ok # TODO Not done yet |
1996 | # results in a xfail test with name 'test 1' and a log of Not done yet |
1997 | - self.tap.write("ok # TODO Not done yet\n") |
1998 | + self.tap.write(_u("ok # TODO Not done yet\n")) |
1999 | self.tap.seek(0) |
2000 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
2001 | self.assertEqual(0, result) |
2002 | - self.assertEqual([ |
2003 | - "test test 1", |
2004 | - "xfail test 1 [", |
2005 | - "Not done yet", |
2006 | - "]", |
2007 | - ], |
2008 | - self.subunit.getvalue().splitlines()) |
2009 | + self.check_events([('status', 'test 1', 'xfail', None, False, |
2010 | + 'tap comment', b'Not done yet', True, 'text/plain; charset=UTF8', |
2011 | + None, None)]) |
2012 | |
2013 | def test_bail_out_errors(self): |
2014 | # A file with line in it |
2015 | # Bail out! COMMENT |
2016 | # is treated as an error |
2017 | - self.tap.write("ok 1 foo\n") |
2018 | - self.tap.write("Bail out! Lifejacket engaged\n") |
2019 | + self.tap.write(_u("ok 1 foo\n")) |
2020 | + self.tap.write(_u("Bail out! Lifejacket engaged\n")) |
2021 | self.tap.seek(0) |
2022 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
2023 | self.assertEqual(0, result) |
2024 | - self.assertEqual([ |
2025 | - "test test 1 foo", |
2026 | - "success test 1 foo", |
2027 | - "test Bail out! Lifejacket engaged", |
2028 | - "error Bail out! Lifejacket engaged", |
2029 | - ], |
2030 | - self.subunit.getvalue().splitlines()) |
2031 | + self.check_events([ |
2032 | + ('status', 'test 1 foo', 'success', None, False, None, None, True, |
2033 | + None, None, None), |
2034 | + ('status', 'Bail out! Lifejacket engaged', 'fail', None, False, |
2035 | + None, None, True, None, None, None)]) |
2036 | |
2037 | def test_missing_test_at_end_with_plan_adds_error(self): |
2038 | # A file |
2039 | @@ -224,23 +190,20 @@ |
2040 | # ok first test |
2041 | # not ok third test |
2042 | # results in three tests, with the third being created |
2043 | - self.tap.write('1..3\n') |
2044 | - self.tap.write('ok first test\n') |
2045 | - self.tap.write('not ok second test\n') |
2046 | + self.tap.write(_u('1..3\n')) |
2047 | + self.tap.write(_u('ok first test\n')) |
2048 | + self.tap.write(_u('not ok second test\n')) |
2049 | self.tap.seek(0) |
2050 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
2051 | self.assertEqual(0, result) |
2052 | - self.assertEqual([ |
2053 | - 'test test 1 first test', |
2054 | - 'success test 1 first test', |
2055 | - 'test test 2 second test', |
2056 | - 'failure test 2 second test', |
2057 | - 'test test 3', |
2058 | - 'error test 3 [', |
2059 | - 'test missing from TAP output', |
2060 | - ']', |
2061 | - ], |
2062 | - self.subunit.getvalue().splitlines()) |
2063 | + self.check_events([ |
2064 | + ('status', 'test 1 first test', 'success', None, False, None, |
2065 | + None, True, None, None, None), |
2066 | + ('status', 'test 2 second test', 'fail', None, False, None, None, |
2067 | + True, None, None, None), |
2068 | + ('status', 'test 3', 'fail', None, False, 'tap meta', |
2069 | + b'test missing from TAP output', True, 'text/plain; charset=UTF8', |
2070 | + None, None)]) |
2071 | |
2072 | def test_missing_test_with_plan_adds_error(self): |
2073 | # A file |
2074 | @@ -248,45 +211,39 @@ |
2075 | # ok first test |
2076 | # not ok 3 third test |
2077 | # results in three tests, with the second being created |
2078 | - self.tap.write('1..3\n') |
2079 | - self.tap.write('ok first test\n') |
2080 | - self.tap.write('not ok 3 third test\n') |
2081 | + self.tap.write(_u('1..3\n')) |
2082 | + self.tap.write(_u('ok first test\n')) |
2083 | + self.tap.write(_u('not ok 3 third test\n')) |
2084 | self.tap.seek(0) |
2085 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
2086 | self.assertEqual(0, result) |
2087 | - self.assertEqual([ |
2088 | - 'test test 1 first test', |
2089 | - 'success test 1 first test', |
2090 | - 'test test 2', |
2091 | - 'error test 2 [', |
2092 | - 'test missing from TAP output', |
2093 | - ']', |
2094 | - 'test test 3 third test', |
2095 | - 'failure test 3 third test', |
2096 | - ], |
2097 | - self.subunit.getvalue().splitlines()) |
2098 | + self.check_events([ |
2099 | + ('status', 'test 1 first test', 'success', None, False, None, None, |
2100 | + True, None, None, None), |
2101 | + ('status', 'test 2', 'fail', None, False, 'tap meta', |
2102 | + b'test missing from TAP output', True, 'text/plain; charset=UTF8', |
2103 | + None, None), |
2104 | + ('status', 'test 3 third test', 'fail', None, False, None, None, |
2105 | + True, None, None, None)]) |
2106 | |
2107 | def test_missing_test_no_plan_adds_error(self): |
2108 | # A file |
2109 | # ok first test |
2110 | # not ok 3 third test |
2111 | # results in three tests, with the second being created |
2112 | - self.tap.write('ok first test\n') |
2113 | - self.tap.write('not ok 3 third test\n') |
2114 | + self.tap.write(_u('ok first test\n')) |
2115 | + self.tap.write(_u('not ok 3 third test\n')) |
2116 | self.tap.seek(0) |
2117 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
2118 | self.assertEqual(0, result) |
2119 | - self.assertEqual([ |
2120 | - 'test test 1 first test', |
2121 | - 'success test 1 first test', |
2122 | - 'test test 2', |
2123 | - 'error test 2 [', |
2124 | - 'test missing from TAP output', |
2125 | - ']', |
2126 | - 'test test 3 third test', |
2127 | - 'failure test 3 third test', |
2128 | - ], |
2129 | - self.subunit.getvalue().splitlines()) |
2130 | + self.check_events([ |
2131 | + ('status', 'test 1 first test', 'success', None, False, None, None, |
2132 | + True, None, None, None), |
2133 | + ('status', 'test 2', 'fail', None, False, 'tap meta', |
2134 | + b'test missing from TAP output', True, 'text/plain; charset=UTF8', |
2135 | + None, None), |
2136 | + ('status', 'test 3 third test', 'fail', None, False, None, None, |
2137 | + True, None, None, None)]) |
2138 | |
2139 | def test_four_tests_in_a_row_trailing_plan(self): |
2140 | # A file |
2141 | @@ -296,25 +253,23 @@ |
2142 | # not ok 4 - fourth |
2143 | # 1..4 |
2144 | # results in four tests numbered and named |
2145 | - self.tap.write('ok 1 - first test in a script with trailing plan\n') |
2146 | - self.tap.write('not ok 2 - second\n') |
2147 | - self.tap.write('ok 3 - third\n') |
2148 | - self.tap.write('not ok 4 - fourth\n') |
2149 | - self.tap.write('1..4\n') |
2150 | + self.tap.write(_u('ok 1 - first test in a script with trailing plan\n')) |
2151 | + self.tap.write(_u('not ok 2 - second\n')) |
2152 | + self.tap.write(_u('ok 3 - third\n')) |
2153 | + self.tap.write(_u('not ok 4 - fourth\n')) |
2154 | + self.tap.write(_u('1..4\n')) |
2155 | self.tap.seek(0) |
2156 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
2157 | self.assertEqual(0, result) |
2158 | - self.assertEqual([ |
2159 | - 'test test 1 - first test in a script with trailing plan', |
2160 | - 'success test 1 - first test in a script with trailing plan', |
2161 | - 'test test 2 - second', |
2162 | - 'failure test 2 - second', |
2163 | - 'test test 3 - third', |
2164 | - 'success test 3 - third', |
2165 | - 'test test 4 - fourth', |
2166 | - 'failure test 4 - fourth' |
2167 | - ], |
2168 | - self.subunit.getvalue().splitlines()) |
2169 | + self.check_events([ |
2170 | + ('status', 'test 1 - first test in a script with trailing plan', |
2171 | + 'success', None, False, None, None, True, None, None, None), |
2172 | + ('status', 'test 2 - second', 'fail', None, False, None, None, |
2173 | + True, None, None, None), |
2174 | + ('status', 'test 3 - third', 'success', None, False, None, None, |
2175 | + True, None, None, None), |
2176 | + ('status', 'test 4 - fourth', 'fail', None, False, None, None, |
2177 | + True, None, None, None)]) |
2178 | |
2179 | def test_four_tests_in_a_row_with_plan(self): |
2180 | # A file |
2181 | @@ -324,25 +279,23 @@ |
2182 | # ok 3 - third |
2183 | # not ok 4 - fourth |
2184 | # results in four tests numbered and named |
2185 | - self.tap.write('1..4\n') |
2186 | - self.tap.write('ok 1 - first test in a script with a plan\n') |
2187 | - self.tap.write('not ok 2 - second\n') |
2188 | - self.tap.write('ok 3 - third\n') |
2189 | - self.tap.write('not ok 4 - fourth\n') |
2190 | + self.tap.write(_u('1..4\n')) |
2191 | + self.tap.write(_u('ok 1 - first test in a script with a plan\n')) |
2192 | + self.tap.write(_u('not ok 2 - second\n')) |
2193 | + self.tap.write(_u('ok 3 - third\n')) |
2194 | + self.tap.write(_u('not ok 4 - fourth\n')) |
2195 | self.tap.seek(0) |
2196 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
2197 | self.assertEqual(0, result) |
2198 | - self.assertEqual([ |
2199 | - 'test test 1 - first test in a script with a plan', |
2200 | - 'success test 1 - first test in a script with a plan', |
2201 | - 'test test 2 - second', |
2202 | - 'failure test 2 - second', |
2203 | - 'test test 3 - third', |
2204 | - 'success test 3 - third', |
2205 | - 'test test 4 - fourth', |
2206 | - 'failure test 4 - fourth' |
2207 | - ], |
2208 | - self.subunit.getvalue().splitlines()) |
2209 | + self.check_events([ |
2210 | + ('status', 'test 1 - first test in a script with a plan', |
2211 | + 'success', None, False, None, None, True, None, None, None), |
2212 | + ('status', 'test 2 - second', 'fail', None, False, None, None, |
2213 | + True, None, None, None), |
2214 | + ('status', 'test 3 - third', 'success', None, False, None, None, |
2215 | + True, None, None, None), |
2216 | + ('status', 'test 4 - fourth', 'fail', None, False, None, None, |
2217 | + True, None, None, None)]) |
2218 | |
2219 | def test_four_tests_in_a_row_no_plan(self): |
2220 | # A file |
2221 | @@ -351,46 +304,43 @@ |
2222 | # ok 3 - third |
2223 | # not ok 4 - fourth |
2224 | # results in four tests numbered and named |
2225 | - self.tap.write('ok 1 - first test in a script with no plan at all\n') |
2226 | - self.tap.write('not ok 2 - second\n') |
2227 | - self.tap.write('ok 3 - third\n') |
2228 | - self.tap.write('not ok 4 - fourth\n') |
2229 | + self.tap.write(_u('ok 1 - first test in a script with no plan at all\n')) |
2230 | + self.tap.write(_u('not ok 2 - second\n')) |
2231 | + self.tap.write(_u('ok 3 - third\n')) |
2232 | + self.tap.write(_u('not ok 4 - fourth\n')) |
2233 | self.tap.seek(0) |
2234 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
2235 | self.assertEqual(0, result) |
2236 | - self.assertEqual([ |
2237 | - 'test test 1 - first test in a script with no plan at all', |
2238 | - 'success test 1 - first test in a script with no plan at all', |
2239 | - 'test test 2 - second', |
2240 | - 'failure test 2 - second', |
2241 | - 'test test 3 - third', |
2242 | - 'success test 3 - third', |
2243 | - 'test test 4 - fourth', |
2244 | - 'failure test 4 - fourth' |
2245 | - ], |
2246 | - self.subunit.getvalue().splitlines()) |
2247 | + self.check_events([ |
2248 | + ('status', 'test 1 - first test in a script with no plan at all', |
2249 | + 'success', None, False, None, None, True, None, None, None), |
2250 | + ('status', 'test 2 - second', 'fail', None, False, None, None, |
2251 | + True, None, None, None), |
2252 | + ('status', 'test 3 - third', 'success', None, False, None, None, |
2253 | + True, None, None, None), |
2254 | + ('status', 'test 4 - fourth', 'fail', None, False, None, None, |
2255 | + True, None, None, None)]) |
2256 | |
2257 | def test_todo_and_skip(self): |
2258 | # A file |
2259 | # not ok 1 - a fail but # TODO but is TODO |
2260 | # not ok 2 - another fail # SKIP instead |
2261 | # results in two tests, numbered and commented. |
2262 | - self.tap.write("not ok 1 - a fail but # TODO but is TODO\n") |
2263 | - self.tap.write("not ok 2 - another fail # SKIP instead\n") |
2264 | + self.tap.write(_u("not ok 1 - a fail but # TODO but is TODO\n")) |
2265 | + self.tap.write(_u("not ok 2 - another fail # SKIP instead\n")) |
2266 | self.tap.seek(0) |
2267 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
2268 | self.assertEqual(0, result) |
2269 | - self.assertEqual([ |
2270 | - 'test test 1 - a fail but', |
2271 | - 'xfail test 1 - a fail but [', |
2272 | - 'but is TODO', |
2273 | - ']', |
2274 | - 'test test 2 - another fail', |
2275 | - 'skip test 2 - another fail [', |
2276 | - 'instead', |
2277 | - ']', |
2278 | - ], |
2279 | - self.subunit.getvalue().splitlines()) |
2280 | + self.subunit.seek(0) |
2281 | + events = StreamResult() |
2282 | + subunit.ByteStreamToStreamResult(self.subunit).run(events) |
2283 | + self.check_events([ |
2284 | + ('status', 'test 1 - a fail but', 'xfail', None, False, |
2285 | + 'tap comment', b'but is TODO', True, 'text/plain; charset=UTF8', |
2286 | + None, None), |
2287 | + ('status', 'test 2 - another fail', 'skip', None, False, |
2288 | + 'tap comment', b'instead', True, 'text/plain; charset=UTF8', |
2289 | + None, None)]) |
2290 | |
2291 | def test_leading_comments_add_to_next_test_log(self): |
2292 | # A file |
2293 | @@ -399,21 +349,17 @@ |
2294 | # ok |
2295 | # results in a single test with the comment included |
2296 | # in the first test and not the second. |
2297 | - self.tap.write("# comment\n") |
2298 | - self.tap.write("ok\n") |
2299 | - self.tap.write("ok\n") |
2300 | + self.tap.write(_u("# comment\n")) |
2301 | + self.tap.write(_u("ok\n")) |
2302 | + self.tap.write(_u("ok\n")) |
2303 | self.tap.seek(0) |
2304 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
2305 | self.assertEqual(0, result) |
2306 | - self.assertEqual([ |
2307 | - 'test test 1', |
2308 | - 'success test 1 [', |
2309 | - '# comment', |
2310 | - ']', |
2311 | - 'test test 2', |
2312 | - 'success test 2', |
2313 | - ], |
2314 | - self.subunit.getvalue().splitlines()) |
2315 | + self.check_events([ |
2316 | + ('status', 'test 1', 'success', None, False, 'tap comment', |
2317 | + b'# comment', True, 'text/plain; charset=UTF8', None, None), |
2318 | + ('status', 'test 2', 'success', None, False, None, None, True, |
2319 | + None, None, None)]) |
2320 | |
2321 | def test_trailing_comments_are_included_in_last_test_log(self): |
2322 | # A file |
2323 | @@ -422,21 +368,23 @@ |
2324 | # # comment |
2325 | # results in a two tests, with the second having the comment |
2326 | # attached to its log. |
2327 | - self.tap.write("ok\n") |
2328 | - self.tap.write("ok\n") |
2329 | - self.tap.write("# comment\n") |
2330 | + self.tap.write(_u("ok\n")) |
2331 | + self.tap.write(_u("ok\n")) |
2332 | + self.tap.write(_u("# comment\n")) |
2333 | self.tap.seek(0) |
2334 | result = subunit.TAP2SubUnit(self.tap, self.subunit) |
2335 | self.assertEqual(0, result) |
2336 | - self.assertEqual([ |
2337 | - 'test test 1', |
2338 | - 'success test 1', |
2339 | - 'test test 2', |
2340 | - 'success test 2 [', |
2341 | - '# comment', |
2342 | - ']', |
2343 | - ], |
2344 | - self.subunit.getvalue().splitlines()) |
2345 | + self.check_events([ |
2346 | + ('status', 'test 1', 'success', None, False, None, None, True, |
2347 | + None, None, None), |
2348 | + ('status', 'test 2', 'success', None, False, 'tap comment', |
2349 | + b'# comment', True, 'text/plain; charset=UTF8', None, None)]) |
2350 | + |
2351 | + def check_events(self, events): |
2352 | + self.subunit.seek(0) |
2353 | + eventstream = StreamResult() |
2354 | + subunit.ByteStreamToStreamResult(self.subunit).run(eventstream) |
2355 | + self.assertEqual(events, eventstream._events) |
2356 | |
2357 | |
2358 | def test_suite(): |
2359 | |
2360 | === added file 'python/subunit/tests/test_test_protocol2.py' |
2361 | --- python/subunit/tests/test_test_protocol2.py 1970-01-01 00:00:00 +0000 |
2362 | +++ python/subunit/tests/test_test_protocol2.py 2013-03-31 05:51:20 +0000 |
2363 | @@ -0,0 +1,415 @@ |
2364 | +# |
2365 | +# subunit: extensions to Python unittest to get test results from subprocesses. |
2366 | +# Copyright (C) 2013 Robert Collins <robertc@robertcollins.net> |
2367 | +# |
2368 | +# Licensed under either the Apache License, Version 2.0 or the BSD 3-clause |
2369 | +# license at the users choice. A copy of both licenses are available in the |
2370 | +# project source as Apache-2.0 and BSD. You may not use this file except in |
2371 | +# compliance with one of these two licences. |
2372 | +# |
2373 | +# Unless required by applicable law or agreed to in writing, software |
2374 | +# distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT |
2375 | +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the |
2376 | +# license you chose for the specific language governing permissions and |
2377 | +# limitations under that license. |
2378 | +# |
2379 | + |
2380 | +from io import BytesIO |
2381 | +import datetime |
2382 | + |
2383 | +from testtools import TestCase |
2384 | +from testtools.matchers import HasLength |
2385 | +from testtools.tests.test_testresult import TestStreamResultContract |
2386 | +from testtools.testresult.doubles import StreamResult |
2387 | + |
2388 | +import subunit |
2389 | +import subunit.iso8601 as iso8601 |
2390 | + |
2391 | +CONSTANT_ENUM = b'\xb3)\x01\x0c\x03foo\x08U_\x1b' |
2392 | +CONSTANT_INPROGRESS = b'\xb3)\x02\x0c\x03foo\x8e\xc1-\xb5' |
2393 | +CONSTANT_SUCCESS = b'\xb3)\x03\x0c\x03fooE\x9d\xfe\x10' |
2394 | +CONSTANT_UXSUCCESS = b'\xb3)\x04\x0c\x03fooX\x98\xce\xa8' |
2395 | +CONSTANT_SKIP = b'\xb3)\x05\x0c\x03foo\x93\xc4\x1d\r' |
2396 | +CONSTANT_FAIL = b'\xb3)\x06\x0c\x03foo\x15Po\xa3' |
2397 | +CONSTANT_XFAIL = b'\xb3)\x07\x0c\x03foo\xde\x0c\xbc\x06' |
2398 | +CONSTANT_EOF = b'\xb3!\x10\x08S\x15\x88\xdc' |
2399 | +CONSTANT_FILE_CONTENT = b'\xb3!@\x13\x06barney\x03wooA5\xe3\x8c' |
2400 | +CONSTANT_MIME = b'\xb3! #\x1aapplication/foo; charset=1x3Q\x15' |
2401 | +CONSTANT_TIMESTAMP = b'\xb3+\x03\x13<\x17T\xcf\x80\xaf\xc8\x03barI\x96>-' |
2402 | +CONSTANT_ROUTE_CODE = b'\xb3-\x03\x13\x03bar\x06source\x9cY9\x19' |
2403 | +CONSTANT_RUNNABLE = b'\xb3(\x03\x0c\x03foo\xe3\xea\xf5\xa4' |
2404 | +CONSTANT_TAGS = b'\xb3)\x80\x15\x03bar\x02\x03foo\x03barTHn\xb4' |
2405 | + |
2406 | + |
2407 | +class TestStreamResultToBytesContract(TestCase, TestStreamResultContract): |
2408 | + """Check that StreamResult behaves as testtools expects.""" |
2409 | + |
2410 | + def _make_result(self): |
2411 | + return subunit.StreamResultToBytes(BytesIO()) |
2412 | + |
2413 | + |
2414 | +class TestStreamResultToBytes(TestCase): |
2415 | + |
2416 | + def _make_result(self): |
2417 | + output = BytesIO() |
2418 | + return subunit.StreamResultToBytes(output), output |
2419 | + |
2420 | + def test_numbers(self): |
2421 | + result = subunit.StreamResultToBytes(BytesIO()) |
2422 | + packet = [] |
2423 | + self.assertRaises(Exception, result._write_number, -1, packet) |
2424 | + self.assertEqual([], packet) |
2425 | + result._write_number(0, packet) |
2426 | + self.assertEqual([b'\x00'], packet) |
2427 | + del packet[:] |
2428 | + result._write_number(63, packet) |
2429 | + self.assertEqual([b'\x3f'], packet) |
2430 | + del packet[:] |
2431 | + result._write_number(64, packet) |
2432 | + self.assertEqual([b'\x40\x40'], packet) |
2433 | + del packet[:] |
2434 | + result._write_number(16383, packet) |
2435 | + self.assertEqual([b'\x7f\xff'], packet) |
2436 | + del packet[:] |
2437 | + result._write_number(16384, packet) |
2438 | + self.assertEqual([b'\x80\x40', b'\x00'], packet) |
2439 | + del packet[:] |
2440 | + result._write_number(4194303, packet) |
2441 | + self.assertEqual([b'\xbf\xff', b'\xff'], packet) |
2442 | + del packet[:] |
2443 | + result._write_number(4194304, packet) |
2444 | + self.assertEqual([b'\xc0\x40\x00\x00'], packet) |
2445 | + del packet[:] |
2446 | + result._write_number(1073741823, packet) |
2447 | + self.assertEqual([b'\xff\xff\xff\xff'], packet) |
2448 | + del packet[:] |
2449 | + self.assertRaises(Exception, result._write_number, 1073741824, packet) |
2450 | + self.assertEqual([], packet) |
2451 | + |
2452 | + def test_volatile_length(self): |
2453 | + # if the length of the packet data before the length itself is |
2454 | + # considered is right on the boundary for length's variable length |
2455 | + # encoding, it is easy to get the length wrong by not accounting for |
2456 | + # length itself. |
2457 | + # that is, the encoder has to ensure that length == sum (length_of_rest |
2458 | + # + length_of_length) |
2459 | + result, output = self._make_result() |
2460 | + # 1 byte short: |
2461 | + result.status(file_name="", file_bytes=b'\xff'*0) |
2462 | + self.assertThat(output.getvalue(), HasLength(10)) |
2463 | + self.assertEqual(b'\x0a', output.getvalue()[3:4]) |
2464 | + output.seek(0) |
2465 | + output.truncate() |
2466 | + # 1 byte long: |
2467 | + result.status(file_name="", file_bytes=b'\xff'*53) |
2468 | + self.assertThat(output.getvalue(), HasLength(63)) |
2469 | + self.assertEqual(b'\x3f', output.getvalue()[3:4]) |
2470 | + output.seek(0) |
2471 | + output.truncate() |
2472 | + # 2 bytes short |
2473 | + result.status(file_name="", file_bytes=b'\xff'*54) |
2474 | + self.assertThat(output.getvalue(), HasLength(65)) |
2475 | + self.assertEqual(b'\x40\x41', output.getvalue()[3:5]) |
2476 | + output.seek(0) |
2477 | + output.truncate() |
2478 | + # 2 bytes long |
2479 | + result.status(file_name="", file_bytes=b'\xff'*16371) |
2480 | + self.assertThat(output.getvalue(), HasLength(16383)) |
2481 | + self.assertEqual(b'\x7f\xff', output.getvalue()[3:5]) |
2482 | + output.seek(0) |
2483 | + output.truncate() |
2484 | + # 3 bytes short |
2485 | + result.status(file_name="", file_bytes=b'\xff'*16372) |
2486 | + self.assertThat(output.getvalue(), HasLength(16385)) |
2487 | + self.assertEqual(b'\x80\x40\x01', output.getvalue()[3:6]) |
2488 | + output.seek(0) |
2489 | + output.truncate() |
2490 | + # 3 bytes long |
2491 | + result.status(file_name="", file_bytes=b'\xff'*4194289) |
2492 | + self.assertThat(output.getvalue(), HasLength(4194303)) |
2493 | + self.assertEqual(b'\xbf\xff\xff', output.getvalue()[3:6]) |
2494 | + output.seek(0) |
2495 | + output.truncate() |
2496 | + self.assertRaises(Exception, result.status, file_name="", |
2497 | + file_bytes=b'\xff'*4194290) |
2498 | + |
2499 | + def test_trivial_enumeration(self): |
2500 | + result, output = self._make_result() |
2501 | + result.status("foo", 'exists') |
2502 | + self.assertEqual(CONSTANT_ENUM, output.getvalue()) |
2503 | + |
2504 | + def test_inprogress(self): |
2505 | + result, output = self._make_result() |
2506 | + result.status("foo", 'inprogress') |
2507 | + self.assertEqual(CONSTANT_INPROGRESS, output.getvalue()) |
2508 | + |
2509 | + def test_success(self): |
2510 | + result, output = self._make_result() |
2511 | + result.status("foo", 'success') |
2512 | + self.assertEqual(CONSTANT_SUCCESS, output.getvalue()) |
2513 | + |
2514 | + def test_uxsuccess(self): |
2515 | + result, output = self._make_result() |
2516 | + result.status("foo", 'uxsuccess') |
2517 | + self.assertEqual(CONSTANT_UXSUCCESS, output.getvalue()) |
2518 | + |
2519 | + def test_skip(self): |
2520 | + result, output = self._make_result() |
2521 | + result.status("foo", 'skip') |
2522 | + self.assertEqual(CONSTANT_SKIP, output.getvalue()) |
2523 | + |
2524 | + def test_fail(self): |
2525 | + result, output = self._make_result() |
2526 | + result.status("foo", 'fail') |
2527 | + self.assertEqual(CONSTANT_FAIL, output.getvalue()) |
2528 | + |
2529 | + def test_xfail(self): |
2530 | + result, output = self._make_result() |
2531 | + result.status("foo", 'xfail') |
2532 | + self.assertEqual(CONSTANT_XFAIL, output.getvalue()) |
2533 | + |
2534 | + def test_unknown_status(self): |
2535 | + result, output = self._make_result() |
2536 | + self.assertRaises(Exception, result.status, "foo", 'boo') |
2537 | + self.assertEqual(b'', output.getvalue()) |
2538 | + |
2539 | + def test_eof(self): |
2540 | + result, output = self._make_result() |
2541 | + result.status(eof=True) |
2542 | + self.assertEqual(CONSTANT_EOF, output.getvalue()) |
2543 | + |
2544 | + def test_file_content(self): |
2545 | + result, output = self._make_result() |
2546 | + result.status(file_name="barney", file_bytes=b"woo") |
2547 | + self.assertEqual(CONSTANT_FILE_CONTENT, output.getvalue()) |
2548 | + |
2549 | + def test_mime(self): |
2550 | + result, output = self._make_result() |
2551 | + result.status(mime_type="application/foo; charset=1") |
2552 | + self.assertEqual(CONSTANT_MIME, output.getvalue()) |
2553 | + |
2554 | + def test_route_code(self): |
2555 | + result, output = self._make_result() |
2556 | + result.status(test_id="bar", test_status='success', |
2557 | + route_code="source") |
2558 | + self.assertEqual(CONSTANT_ROUTE_CODE, output.getvalue()) |
2559 | + |
2560 | + def test_runnable(self): |
2561 | + result, output = self._make_result() |
2562 | + result.status("foo", 'success', runnable=False) |
2563 | + self.assertEqual(CONSTANT_RUNNABLE, output.getvalue()) |
2564 | + |
2565 | + def test_tags(self): |
2566 | + result, output = self._make_result() |
2567 | + result.status(test_id="bar", test_tags=set(['foo', 'bar'])) |
2568 | + self.assertEqual(CONSTANT_TAGS, output.getvalue()) |
2569 | + |
2570 | + def test_timestamp(self): |
2571 | + timestamp = datetime.datetime(2001, 12, 12, 12, 59, 59, 45, |
2572 | + iso8601.Utc()) |
2573 | + result, output = self._make_result() |
2574 | + result.status(test_id="bar", test_status='success', timestamp=timestamp) |
2575 | + self.assertEqual(CONSTANT_TIMESTAMP, output.getvalue()) |
2576 | + |
2577 | + |
2578 | +class TestByteStreamToStreamResult(TestCase): |
2579 | + |
2580 | + def test_non_subunit_encapsulated(self): |
2581 | + source = BytesIO(b"foo\nbar\n") |
2582 | + result = StreamResult() |
2583 | + subunit.ByteStreamToStreamResult( |
2584 | + source, non_subunit_name="stdout").run(result) |
2585 | + self.assertEqual([ |
2586 | + ('status', None, None, None, True, 'stdout', b'f', False, None, None, None), |
2587 | + ('status', None, None, None, True, 'stdout', b'o', False, None, None, None), |
2588 | + ('status', None, None, None, True, 'stdout', b'o', False, None, None, None), |
2589 | + ('status', None, None, None, True, 'stdout', b'\n', False, None, None, None), |
2590 | + ('status', None, None, None, True, 'stdout', b'b', False, None, None, None), |
2591 | + ('status', None, None, None, True, 'stdout', b'a', False, None, None, None), |
2592 | + ('status', None, None, None, True, 'stdout', b'r', False, None, None, None), |
2593 | + ('status', None, None, None, True, 'stdout', b'\n', False, None, None, None), |
2594 | + ], result._events) |
2595 | + self.assertEqual(b'', source.read()) |
2596 | + |
2597 | + def test_signature_middle_utf8_char(self): |
2598 | + utf8_bytes = b'\xe3\xb3\x8a' |
2599 | + source = BytesIO(utf8_bytes) |
2600 | + # Should be treated as one character (it is u'\u3cca') and wrapped |
2601 | + result = StreamResult() |
2602 | + subunit.ByteStreamToStreamResult( |
2603 | + source, non_subunit_name="stdout").run( |
2604 | + result) |
2605 | + self.assertEqual([ |
2606 | + ('status', None, None, None, True, 'stdout', b'\xe3', False, None, None, None), |
2607 | + ('status', None, None, None, True, 'stdout', b'\xb3', False, None, None, None), |
2608 | + ('status', None, None, None, True, 'stdout', b'\x8a', False, None, None, None), |
2609 | + ], result._events) |
2610 | + |
2611 | + def test_non_subunit_disabled_raises(self): |
2612 | + source = BytesIO(b"foo\nbar\n") |
2613 | + result = StreamResult() |
2614 | + case = subunit.ByteStreamToStreamResult(source) |
2615 | + e = self.assertRaises(Exception, case.run, result) |
2616 | + self.assertEqual(b'f', e.args[1]) |
2617 | + self.assertEqual(b'oo\nbar\n', source.read()) |
2618 | + self.assertEqual([], result._events) |
2619 | + |
2620 | + def test_trivial_enumeration(self): |
2621 | + source = BytesIO(CONSTANT_ENUM) |
2622 | + result = StreamResult() |
2623 | + subunit.ByteStreamToStreamResult( |
2624 | + source, non_subunit_name="stdout").run(result) |
2625 | + self.assertEqual(b'', source.read()) |
2626 | + self.assertEqual([ |
2627 | + ('status', 'foo', 'exists', None, True, None, None, False, None, None, None), |
2628 | + ], result._events) |
2629 | + |
2630 | + def test_multiple_events(self): |
2631 | + source = BytesIO(CONSTANT_ENUM + CONSTANT_ENUM) |
2632 | + result = StreamResult() |
2633 | + subunit.ByteStreamToStreamResult( |
2634 | + source, non_subunit_name="stdout").run(result) |
2635 | + self.assertEqual(b'', source.read()) |
2636 | + self.assertEqual([ |
2637 | + ('status', 'foo', 'exists', None, True, None, None, False, None, None, None), |
2638 | + ('status', 'foo', 'exists', None, True, None, None, False, None, None, None), |
2639 | + ], result._events) |
2640 | + |
2641 | + def test_inprogress(self): |
2642 | + self.check_event(CONSTANT_INPROGRESS, 'inprogress') |
2643 | + |
2644 | + def test_success(self): |
2645 | + self.check_event(CONSTANT_SUCCESS, 'success') |
2646 | + |
2647 | + def test_uxsuccess(self): |
2648 | + self.check_event(CONSTANT_UXSUCCESS, 'uxsuccess') |
2649 | + |
2650 | + def test_skip(self): |
2651 | + self.check_event(CONSTANT_SKIP, 'skip') |
2652 | + |
2653 | + def test_fail(self): |
2654 | + self.check_event(CONSTANT_FAIL, 'fail') |
2655 | + |
2656 | + def test_xfail(self): |
2657 | + self.check_event(CONSTANT_XFAIL, 'xfail') |
2658 | + |
2659 | + def check_events(self, source_bytes, events): |
2660 | + source = BytesIO(source_bytes) |
2661 | + result = StreamResult() |
2662 | + subunit.ByteStreamToStreamResult( |
2663 | + source, non_subunit_name="stdout").run(result) |
2664 | + self.assertEqual(b'', source.read()) |
2665 | + self.assertEqual(events, result._events) |
2666 | + |
2667 | + def check_event(self, source_bytes, test_status=None, test_id="foo", |
2668 | + route_code=None, timestamp=None, tags=None, mime_type=None, |
2669 | + file_name=None, file_bytes=None, eof=False, runnable=True): |
2670 | + event = self._event(test_id=test_id, test_status=test_status, |
2671 | + tags=tags, runnable=runnable, file_name=file_name, |
2672 | + file_bytes=file_bytes, eof=eof, mime_type=mime_type, |
2673 | + route_code=route_code, timestamp=timestamp) |
2674 | + self.check_events(source_bytes, [event]) |
2675 | + |
2676 | + def _event(self, test_status=None, test_id=None, route_code=None, |
2677 | + timestamp=None, tags=None, mime_type=None, file_name=None, |
2678 | + file_bytes=None, eof=False, runnable=True): |
2679 | + return ('status', test_id, test_status, tags, runnable, file_name, |
2680 | + file_bytes, eof, mime_type, route_code, timestamp) |
2681 | + |
2682 | + def test_eof(self): |
2683 | + self.check_event(CONSTANT_EOF, test_id=None, eof=True) |
2684 | + |
2685 | + def test_file_content(self): |
2686 | + self.check_event(CONSTANT_FILE_CONTENT, |
2687 | + test_id=None, file_name="barney", file_bytes=b"woo") |
2688 | + |
2689 | + def test_file_content_length_into_checksum(self): |
2690 | + # A bad file content length which creeps into the checksum. |
2691 | + bad_file_length_content = b'\xb3!@\x13\x06barney\x04woo\xdc\xe2\xdb\x35' |
2692 | + self.check_events(bad_file_length_content, [ |
2693 | + self._event(test_id="subunit.parser", eof=True, |
2694 | + file_name="Packet data", file_bytes=bad_file_length_content), |
2695 | + self._event(test_id="subunit.parser", test_status="fail", eof=True, |
2696 | + file_name="Parser Error", |
2697 | + file_bytes=b"File content extends past end of packet: claimed 4 bytes, 3 available"), |
2698 | + ]) |
2699 | + |
2700 | + def test_packet_length_4_word_varint(self): |
2701 | + packet_data = b'\xb3!@\xc0\x00\x11' |
2702 | + self.check_events(packet_data, [ |
2703 | + self._event(test_id="subunit.parser", eof=True, |
2704 | + file_name="Packet data", file_bytes=packet_data), |
2705 | + self._event(test_id="subunit.parser", test_status="fail", eof=True, |
2706 | + file_name="Parser Error", |
2707 | + file_bytes=b"3 byte maximum given but 4 byte value found."), |
2708 | + ]) |
2709 | + |
2710 | + def test_mime(self): |
2711 | + self.check_event(CONSTANT_MIME, |
2712 | + test_id=None, mime_type='application/foo; charset=1') |
2713 | + |
2714 | + def test_route_code(self): |
2715 | + self.check_event(CONSTANT_ROUTE_CODE, |
2716 | + 'success', route_code="source", test_id="bar") |
2717 | + |
2718 | + def test_runnable(self): |
2719 | + self.check_event(CONSTANT_RUNNABLE, |
2720 | + test_status='success', runnable=False) |
2721 | + |
2722 | + def test_tags(self): |
2723 | + self.check_event(CONSTANT_TAGS, |
2724 | + None, tags=set(['foo', 'bar']), test_id="bar") |
2725 | + |
2726 | + def test_timestamp(self): |
2727 | + timestamp = datetime.datetime(2001, 12, 12, 12, 59, 59, 45, |
2728 | + iso8601.Utc()) |
2729 | + self.check_event(CONSTANT_TIMESTAMP, |
2730 | + 'success', test_id='bar', timestamp=timestamp) |
2731 | + |
2732 | + def test_bad_crc_errors_via_status(self): |
2733 | + file_bytes = CONSTANT_MIME[:-1] + b'\x00' |
2734 | + self.check_events( file_bytes, [ |
2735 | + self._event(test_id="subunit.parser", eof=True, |
2736 | + file_name="Packet data", file_bytes=file_bytes), |
2737 | + self._event(test_id="subunit.parser", test_status="fail", eof=True, |
2738 | + file_name="Parser Error", |
2739 | + file_bytes=b'Bad checksum - calculated (0x78335115), ' |
2740 | + b'stored (0x78335100)'), |
2741 | + ]) |
2742 | + |
2743 | + def test_not_utf8_in_string(self): |
2744 | + file_bytes = CONSTANT_ROUTE_CODE[:5] + b'\xb4' + CONSTANT_ROUTE_CODE[6:-4] + b'\xce\x56\xc6\x17' |
2745 | + self.check_events(file_bytes, [ |
2746 | + self._event(test_id="subunit.parser", eof=True, |
2747 | + file_name="Packet data", file_bytes=file_bytes), |
2748 | + self._event(test_id="subunit.parser", test_status="fail", eof=True, |
2749 | + file_name="Parser Error", |
2750 | + file_bytes=b'UTF8 string at offset 2 is not UTF8'), |
2751 | + ]) |
2752 | + |
2753 | + def test_NULL_in_string(self): |
2754 | + file_bytes = CONSTANT_ROUTE_CODE[:6] + b'\x00' + CONSTANT_ROUTE_CODE[7:-4] + b'\xd7\x41\xac\xfe' |
2755 | + self.check_events(file_bytes, [ |
2756 | + self._event(test_id="subunit.parser", eof=True, |
2757 | + file_name="Packet data", file_bytes=file_bytes), |
2758 | + self._event(test_id="subunit.parser", test_status="fail", eof=True, |
2759 | + file_name="Parser Error", |
2760 | + file_bytes=b'UTF8 string at offset 2 contains NUL byte'), |
2761 | + ]) |
2762 | + |
2763 | + def test_bad_utf8_stringlength(self): |
2764 | + file_bytes = CONSTANT_ROUTE_CODE[:4] + b'\x3f' + CONSTANT_ROUTE_CODE[5:-4] + b'\xbe\x29\xe0\xc2' |
2765 | + self.check_events(file_bytes, [ |
2766 | + self._event(test_id="subunit.parser", eof=True, |
2767 | + file_name="Packet data", file_bytes=file_bytes), |
2768 | + self._event(test_id="subunit.parser", test_status="fail", eof=True, |
2769 | + file_name="Parser Error", |
2770 | + file_bytes=b'UTF8 string at offset 2 extends past end of ' |
2771 | + b'packet: claimed 63 bytes, 10 available'), |
2772 | + ]) |
2773 | + |
2774 | + |
2775 | +def test_suite(): |
2776 | + loader = subunit.tests.TestUtil.TestLoader() |
2777 | + result = loader.loadTestsFromName(__name__) |
2778 | + return result |
2779 | |
2780 | === added file 'python/subunit/v2.py' |
2781 | --- python/subunit/v2.py 1970-01-01 00:00:00 +0000 |
2782 | +++ python/subunit/v2.py 2013-03-31 05:51:20 +0000 |
2783 | @@ -0,0 +1,458 @@ |
2784 | +# |
2785 | +# subunit: extensions to Python unittest to get test results from subprocesses. |
2786 | +# Copyright (C) 2013 Robert Collins <robertc@robertcollins.net> |
2787 | +# |
2788 | +# Licensed under either the Apache License, Version 2.0 or the BSD 3-clause |
2789 | +# license at the users choice. A copy of both licenses are available in the |
2790 | +# project source as Apache-2.0 and BSD. You may not use this file except in |
2791 | +# compliance with one of these two licences. |
2792 | +# |
2793 | +# Unless required by applicable law or agreed to in writing, software |
2794 | +# distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT |
2795 | +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the |
2796 | +# license you chose for the specific language governing permissions and |
2797 | +# limitations under that license. |
2798 | +# |
2799 | + |
2800 | +import codecs |
2801 | +import datetime |
2802 | +from io import UnsupportedOperation |
2803 | +import os |
2804 | +import select |
2805 | +import struct |
2806 | +import zlib |
2807 | + |
2808 | +from extras import safe_hasattr |
2809 | + |
2810 | +import subunit |
2811 | +import subunit.iso8601 as iso8601 |
2812 | + |
2813 | +__all__ = [ |
2814 | + 'ByteStreamToStreamResult', |
2815 | + 'StreamResultToBytes', |
2816 | + ] |
2817 | + |
2818 | +SIGNATURE = b'\xb3' |
2819 | +FMT_8 = '>B' |
2820 | +FMT_16 = '>H' |
2821 | +FMT_24 = '>HB' |
2822 | +FMT_32 = '>I' |
2823 | +FMT_TIMESTAMP = '>II' |
2824 | +FLAG_TEST_ID = 0x0800 |
2825 | +FLAG_ROUTE_CODE = 0x0400 |
2826 | +FLAG_TIMESTAMP = 0x0200 |
2827 | +FLAG_RUNNABLE = 0x0100 |
2828 | +FLAG_TAGS = 0x0080 |
2829 | +FLAG_MIME_TYPE = 0x0020 |
2830 | +FLAG_EOF = 0x0010 |
2831 | +FLAG_FILE_CONTENT = 0x0040 |
2832 | +EPOCH = datetime.datetime.utcfromtimestamp(0).replace(tzinfo=iso8601.Utc()) |
2833 | +NUL_ELEMENT = b'\0'[0] |
2834 | + |
2835 | + |
2836 | +class ParseError(Exception): |
2837 | + """Used to pass error messages within the parser.""" |
2838 | + |
2839 | + |
2840 | +class StreamResultToBytes(object): |
2841 | + """Convert StreamResult API calls to bytes. |
2842 | + |
2843 | + The StreamResult API is defined by testtools.StreamResult. |
2844 | + """ |
2845 | + |
2846 | + status_mask = { |
2847 | + None: 0, |
2848 | + 'exists': 0x1, |
2849 | + 'inprogress': 0x2, |
2850 | + 'success': 0x3, |
2851 | + 'uxsuccess': 0x4, |
2852 | + 'skip': 0x5, |
2853 | + 'fail': 0x6, |
2854 | + 'xfail': 0x7, |
2855 | + } |
2856 | + |
2857 | + zero_b = b'\0'[0] |
2858 | + |
2859 | + def __init__(self, output_stream): |
2860 | + """Create a StreamResultToBytes with output written to output_stream. |
2861 | + |
2862 | + :param output_stream: A file-like object. Must support write(bytes) |
2863 | + and flush() methods. Flush will be called after each write. |
2864 | + The stream will be passed through subunit.make_stream_binary, |
2865 | + to handle regular cases such as stdout. |
2866 | + """ |
2867 | + self.output_stream = subunit.make_stream_binary(output_stream) |
2868 | + |
2869 | + def startTestRun(self): |
2870 | + pass |
2871 | + |
2872 | + def stopTestRun(self): |
2873 | + pass |
2874 | + |
2875 | + def status(self, test_id=None, test_status=None, test_tags=None, |
2876 | + runnable=True, file_name=None, file_bytes=None, eof=False, |
2877 | + mime_type=None, route_code=None, timestamp=None): |
2878 | + self._write_packet(test_id=test_id, test_status=test_status, |
2879 | + test_tags=test_tags, runnable=runnable, file_name=file_name, |
2880 | + file_bytes=file_bytes, eof=eof, mime_type=mime_type, |
2881 | + route_code=route_code, timestamp=timestamp) |
2882 | + |
2883 | + def _write_utf8(self, a_string, packet): |
2884 | + utf8 = a_string.encode('utf-8') |
2885 | + self._write_number(len(utf8), packet) |
2886 | + packet.append(utf8) |
2887 | + |
2888 | + def _write_len16(self, length, packet): |
2889 | + assert length < 65536 |
2890 | + packet.append(struct.pack(FMT_16, length)) |
2891 | + |
2892 | + def _write_number(self, value, packet): |
2893 | + packet.extend(self._encode_number(value)) |
2894 | + |
2895 | + def _encode_number(self, value): |
2896 | + assert value >= 0 |
2897 | + if value < 64: |
2898 | + return [struct.pack(FMT_8, value)] |
2899 | + elif value < 16384: |
2900 | + value = value | 0x4000 |
2901 | + return [struct.pack(FMT_16, value)] |
2902 | + elif value < 4194304: |
2903 | + value = value | 0x800000 |
2904 | + return [struct.pack(FMT_16, value >> 8), |
2905 | + struct.pack(FMT_8, value & 0xff)] |
2906 | + elif value < 1073741824: |
2907 | + value = value | 0xc0000000 |
2908 | + return [struct.pack(FMT_32, value)] |
2909 | + else: |
2910 | + raise ValueError('value too large to encode: %r' % (value,)) |
2911 | + |
2912 | + def _write_packet(self, test_id=None, test_status=None, test_tags=None, |
2913 | + runnable=True, file_name=None, file_bytes=None, eof=False, |
2914 | + mime_type=None, route_code=None, timestamp=None): |
2915 | + packet = [SIGNATURE] |
2916 | + packet.append(b'FF') # placeholder for flags |
2917 | + # placeholder for length, but see below as length is variable. |
2918 | + packet.append(b'') |
2919 | + flags = 0x2000 # Version 0x2 |
2920 | + if timestamp is not None: |
2921 | + flags = flags | FLAG_TIMESTAMP |
2922 | + since_epoch = timestamp - EPOCH |
2923 | + nanoseconds = since_epoch.microseconds * 1000 |
2924 | + seconds = (since_epoch.seconds + since_epoch.days * 24 * 3600) |
2925 | + packet.append(struct.pack(FMT_32, seconds)) |
2926 | + self._write_number(nanoseconds, packet) |
2927 | + if test_id is not None: |
2928 | + flags = flags | FLAG_TEST_ID |
2929 | + self._write_utf8(test_id, packet) |
2930 | + if test_tags: |
2931 | + flags = flags | FLAG_TAGS |
2932 | + self._write_number(len(test_tags), packet) |
2933 | + for tag in test_tags: |
2934 | + self._write_utf8(tag, packet) |
2935 | + if runnable: |
2936 | + flags = flags | FLAG_RUNNABLE |
2937 | + if mime_type: |
2938 | + flags = flags | FLAG_MIME_TYPE |
2939 | + self._write_utf8(mime_type, packet) |
2940 | + if file_name is not None: |
2941 | + flags = flags | FLAG_FILE_CONTENT |
2942 | + self._write_utf8(file_name, packet) |
2943 | + self._write_number(len(file_bytes), packet) |
2944 | + packet.append(file_bytes) |
2945 | + if eof: |
2946 | + flags = flags | FLAG_EOF |
2947 | + if route_code is not None: |
2948 | + flags = flags | FLAG_ROUTE_CODE |
2949 | + self._write_utf8(route_code, packet) |
2950 | + # 0x0008 - not used in v2. |
2951 | + flags = flags | self.status_mask[test_status] |
2952 | + packet[1] = struct.pack(FMT_16, flags) |
2953 | + base_length = sum(map(len, packet)) + 4 |
2954 | + if base_length <= 62: |
2955 | + # one byte to encode length, 62+1 = 63 |
2956 | + length_length = 1 |
2957 | + elif base_length <= 16381: |
2958 | + # two bytes to encode length, 16381+2 = 16383 |
2959 | + length_length = 2 |
2960 | + elif base_length <= 4194300: |
2961 | + # three bytes to encode length, 419430+3=4194303 |
2962 | + length_length = 3 |
2963 | + else: |
2964 | + # Longer than policy: |
2965 | + # TODO: chunk the packet automatically? |
2966 | + # - strip all but file data |
2967 | + # - do 4M chunks of that till done |
2968 | + # - include original data in final chunk. |
2969 | + raise ValueError("Length too long: %r" % base_length) |
2970 | + packet[2:3] = self._encode_number(base_length + length_length) |
2971 | + # We could either do a partial application of crc32 over each chunk |
2972 | + # or a single join to a temp variable then a final join |
2973 | + # or two writes (that python might then split). |
2974 | + # For now, simplest code: join, crc32, join, output |
2975 | + content = b''.join(packet) |
2976 | + self.output_stream.write(content + struct.pack( |
2977 | + FMT_32, zlib.crc32(content) & 0xffffffff)) |
2978 | + self.output_stream.flush() |
2979 | + |
2980 | + |
2981 | +class ByteStreamToStreamResult(object): |
2982 | + """Parse a subunit byte stream. |
2983 | + |
2984 | + Mixed streams that contain non-subunit content is supported when a |
2985 | + non_subunit_name is passed to the contructor. The default is to raise an |
2986 | + error containing the non-subunit byte after it has been read from the |
2987 | + stream. |
2988 | + |
2989 | + Typical use: |
2990 | + |
2991 | + >>> case = ByteStreamToStreamResult(sys.stdin.buffer) |
2992 | + >>> result = StreamResult() |
2993 | + >>> result.startTestRun() |
2994 | + >>> case.run(result) |
2995 | + >>> result.stopTestRun() |
2996 | + """ |
2997 | + |
2998 | + status_lookup = { |
2999 | + 0x0: None, |
3000 | + 0x1: 'exists', |
3001 | + 0x2: 'inprogress', |
3002 | + 0x3: 'success', |
3003 | + 0x4: 'uxsuccess', |
3004 | + 0x5: 'skip', |
3005 | + 0x6: 'fail', |
3006 | + 0x7: 'xfail', |
3007 | + } |
3008 | + |
3009 | + def __init__(self, source, non_subunit_name=None): |
3010 | + """Create a ByteStreamToStreamResult. |
3011 | + |
3012 | + :param source: A file like object to read bytes from. Must support |
3013 | + read(<count>) and return bytes. The file is not closed by |
3014 | + ByteStreamToStreamResult. subunit.make_stream_binary() is |
3015 | + called on the stream to get it into bytes mode. |
3016 | + :param non_subunit_name: If set to non-None, non subunit content |
3017 | + encountered in the stream will be converted into file packets |
3018 | + labelled with this name. |
3019 | + """ |
3020 | + self.non_subunit_name = non_subunit_name |
3021 | + self.source = subunit.make_stream_binary(source) |
3022 | + self.codec = codecs.lookup('utf8').incrementaldecoder() |
3023 | + |
3024 | + def run(self, result): |
3025 | + """Parse source and emit events to result. |
3026 | + |
3027 | + This is a blocking call: it will run until EOF is detected on source. |
3028 | + """ |
3029 | + self.codec.reset() |
3030 | + mid_character = False |
3031 | + while True: |
3032 | + # We're in blocking mode; read one char |
3033 | + content = self.source.read(1) |
3034 | + if not content: |
3035 | + # EOF |
3036 | + return |
3037 | + if not mid_character and content[0] == SIGNATURE[0]: |
3038 | + self._parse_packet(result) |
3039 | + continue |
3040 | + if self.non_subunit_name is None: |
3041 | + raise Exception("Non subunit content", content) |
3042 | + try: |
3043 | + if self.codec.decode(content): |
3044 | + # End of a character |
3045 | + mid_character = False |
3046 | + else: |
3047 | + mid_character = True |
3048 | + except UnicodeDecodeError: |
3049 | + # Bad unicode, not our concern. |
3050 | + mid_character = False |
3051 | + # Aggregate all content that is not subunit until either |
3052 | + # 1MiB is accumulated or 50ms has passed with no input. |
3053 | + # Both are arbitrary amounts intended to give a simple |
3054 | + # balance between efficiency (avoiding death by a thousand |
3055 | + # one-byte packets), buffering (avoiding overlarge state |
3056 | + # being hidden on intermediary nodes) and interactivity |
3057 | + # (when driving a debugger, slow response to typing is |
3058 | + # annoying). |
3059 | + buffered = [content] |
3060 | + while len(buffered[-1]): |
3061 | + try: |
3062 | + self.source.fileno() |
3063 | + except: |
3064 | + # Won't be able to select, fallback to |
3065 | + # one-byte-at-a-time. |
3066 | + break |
3067 | + # Note: this has a very low timeout because with stdin, the |
3068 | + # BufferedIO layer typically has all the content available |
3069 | + # from the stream when e.g. pdb is dropped into, leading to |
3070 | + # select always timing out when in fact we could have read |
3071 | + # (from the buffer layer) - we typically fail to aggregate |
3072 | + # any content on 3.x Pythons. |
3073 | + readable = select.select([self.source], [], [], 0.000001)[0] |
3074 | + if readable: |
3075 | + content = self.source.read(1) |
3076 | + if not len(content): |
3077 | + # EOF - break and emit buffered. |
3078 | + break |
3079 | + if not mid_character and content[0] == SIGNATURE[0]: |
3080 | + # New packet, break, emit buffered, then parse. |
3081 | + break |
3082 | + buffered.append(content) |
3083 | + # Feed into the codec. |
3084 | + try: |
3085 | + if self.codec.decode(content): |
3086 | + # End of a character |
3087 | + mid_character = False |
3088 | + else: |
3089 | + mid_character = True |
3090 | + except UnicodeDecodeError: |
3091 | + # Bad unicode, not our concern. |
3092 | + mid_character = False |
3093 | + if not readable or len(buffered) >= 1048576: |
3094 | + # timeout or too much data, emit what we have. |
3095 | + break |
3096 | + result.status( |
3097 | + file_name=self.non_subunit_name, |
3098 | + file_bytes=b''.join(buffered)) |
3099 | + if mid_character or not len(content) or content[0] != SIGNATURE[0]: |
3100 | + continue |
3101 | + # Otherwise, parse a data packet. |
3102 | + self._parse_packet(result) |
3103 | + |
3104 | + def _parse_packet(self, result): |
3105 | + try: |
3106 | + packet = [SIGNATURE] |
3107 | + self._parse(packet, result) |
3108 | + except ParseError as error: |
3109 | + result.status(test_id="subunit.parser", eof=True, |
3110 | + file_name="Packet data", file_bytes=b''.join(packet)) |
3111 | + result.status(test_id="subunit.parser", test_status='fail', |
3112 | + eof=True, file_name="Parser Error", |
3113 | + file_bytes=(error.args[0]).encode('utf8')) |
3114 | + |
3115 | + def _parse_varint(self, data, pos, max_3_bytes=False): |
3116 | + # because the only incremental IO we do is at the start, and the 32 bit |
3117 | + # CRC means we can always safely read enough to cover any varint, we |
3118 | + # can be sure that there should be enough data - and if not it is an |
3119 | + # error not a normal situation. |
3120 | + data_0 = struct.unpack(FMT_8, data[pos:pos+1])[0] |
3121 | + typeenum = data_0 & 0xc0 |
3122 | + value_0 = data_0 & 0x3f |
3123 | + if typeenum == 0x00: |
3124 | + return value_0, 1 |
3125 | + elif typeenum == 0x40: |
3126 | + data_1 = struct.unpack(FMT_8, data[pos+1:pos+2])[0] |
3127 | + return (value_0 << 8) | data_1, 2 |
3128 | + elif typeenum == 0x80: |
3129 | + data_1 = struct.unpack(FMT_16, data[pos+1:pos+3])[0] |
3130 | + return (value_0 << 16) | data_1, 3 |
3131 | + else: |
3132 | + if max_3_bytes: |
3133 | + raise ParseError('3 byte maximum given but 4 byte value found.') |
3134 | + data_1, data_2 = struct.unpack(FMT_24, data[pos+1:pos+4]) |
3135 | + result = (value_0 << 24) | data_1 << 8 | data_2 |
3136 | + return result, 4 |
3137 | + |
3138 | + def _parse(self, packet, result): |
3139 | + # 2 bytes flags, at most 3 bytes length. |
3140 | + packet.append(self.source.read(5)) |
3141 | + flags = struct.unpack(FMT_16, packet[-1][:2])[0] |
3142 | + length, consumed = self._parse_varint( |
3143 | + packet[-1], 2, max_3_bytes=True) |
3144 | + remainder = self.source.read(length - 6) |
3145 | + if len(remainder) != length - 6: |
3146 | + raise ParseError( |
3147 | + 'Short read - got %d bytes, wanted %d bytes' % ( |
3148 | + len(remainder), length - 6)) |
3149 | + if consumed != 3: |
3150 | + # Avoid having to parse torn values |
3151 | + packet[-1] += remainder |
3152 | + pos = 2 + consumed |
3153 | + else: |
3154 | + # Avoid copying potentially lots of data. |
3155 | + packet.append(remainder) |
3156 | + pos = 0 |
3157 | + crc = zlib.crc32(packet[0]) |
3158 | + for fragment in packet[1:-1]: |
3159 | + crc = zlib.crc32(fragment, crc) |
3160 | + crc = zlib.crc32(packet[-1][:-4], crc) & 0xffffffff |
3161 | + packet_crc = struct.unpack(FMT_32, packet[-1][-4:])[0] |
3162 | + if crc != packet_crc: |
3163 | + # Bad CRC, report it and stop parsing the packet. |
3164 | + raise ParseError( |
3165 | + 'Bad checksum - calculated (0x%x), stored (0x%x)' |
3166 | + % (crc, packet_crc)) |
3167 | + if safe_hasattr(__builtins__, 'memoryview'): |
3168 | + body = memoryview(packet[-1]) |
3169 | + else: |
3170 | + body = packet[-1] |
3171 | + # Discard CRC-32 |
3172 | + body = body[:-4] |
3173 | + # One packet could have both file and status data; the Python API |
3174 | + # presents these separately (perhaps it shouldn't?) |
3175 | + if flags & FLAG_TIMESTAMP: |
3176 | + seconds = struct.unpack(FMT_32, body[pos:pos+4])[0] |
3177 | + nanoseconds, consumed = self._parse_varint(body, pos+4) |
3178 | + pos = pos + 4 + consumed |
3179 | + timestamp = EPOCH + datetime.timedelta( |
3180 | + seconds=seconds, microseconds=nanoseconds/1000) |
3181 | + else: |
3182 | + timestamp = None |
3183 | + if flags & FLAG_TEST_ID: |
3184 | + test_id, pos = self._read_utf8(body, pos) |
3185 | + else: |
3186 | + test_id = None |
3187 | + if flags & FLAG_TAGS: |
3188 | + tag_count, consumed = self._parse_varint(body, pos) |
3189 | + pos += consumed |
3190 | + test_tags = set() |
3191 | + for _ in range(tag_count): |
3192 | + tag, pos = self._read_utf8(body, pos) |
3193 | + test_tags.add(tag) |
3194 | + else: |
3195 | + test_tags = None |
3196 | + if flags & FLAG_MIME_TYPE: |
3197 | + mime_type, pos = self._read_utf8(body, pos) |
3198 | + else: |
3199 | + mime_type = None |
3200 | + if flags & FLAG_FILE_CONTENT: |
3201 | + file_name, pos = self._read_utf8(body, pos) |
3202 | + content_length, consumed = self._parse_varint(body, pos) |
3203 | + pos += consumed |
3204 | + file_bytes = body[pos:pos+content_length] |
3205 | + if len(file_bytes) != content_length: |
3206 | + raise ParseError('File content extends past end of packet: ' |
3207 | + 'claimed %d bytes, %d available' % ( |
3208 | + content_length, len(file_bytes))) |
3209 | + else: |
3210 | + file_name = None |
3211 | + file_bytes = None |
3212 | + if flags & FLAG_ROUTE_CODE: |
3213 | + route_code, pos = self._read_utf8(body, pos) |
3214 | + else: |
3215 | + route_code = None |
3216 | + runnable = bool(flags & FLAG_RUNNABLE) |
3217 | + eof = bool(flags & FLAG_EOF) |
3218 | + test_status = self.status_lookup[flags & 0x0007] |
3219 | + result.status(test_id=test_id, test_status=test_status, |
3220 | + test_tags=test_tags, runnable=runnable, mime_type=mime_type, |
3221 | + eof=eof, file_name=file_name, file_bytes=file_bytes, |
3222 | + route_code=route_code, timestamp=timestamp) |
3223 | + __call__ = run |
3224 | + |
3225 | + def _read_utf8(self, buf, pos): |
3226 | + length, consumed = self._parse_varint(buf, pos) |
3227 | + pos += consumed |
3228 | + utf8_bytes = buf[pos:pos+length] |
3229 | + if length != len(utf8_bytes): |
3230 | + raise ParseError( |
3231 | + 'UTF8 string at offset %d extends past end of packet: ' |
3232 | + 'claimed %d bytes, %d available' % (pos - 2, length, |
3233 | + len(utf8_bytes))) |
3234 | + if NUL_ELEMENT in utf8_bytes: |
3235 | + raise ParseError('UTF8 string at offset %d contains NUL byte' % ( |
3236 | + pos-2,)) |
3237 | + try: |
3238 | + return utf8_bytes.decode('utf-8'), length+pos |
3239 | + except UnicodeDecodeError: |
3240 | + raise ParseError('UTF8 string at offset %d is not UTF8' % (pos-2,)) |
3241 | + |
3242 | |
3243 | === modified file 'setup.py' |
3244 | --- setup.py 2012-12-17 08:12:44 +0000 |
3245 | +++ setup.py 2013-03-31 05:51:20 +0000 |
3246 | @@ -9,6 +9,7 @@ |
3247 | else: |
3248 | extra = { |
3249 | 'install_requires': [ |
3250 | + 'extras', |
3251 | 'testtools>=0.9.23', |
3252 | ] |
3253 | } |
3254 | @@ -49,6 +50,8 @@ |
3255 | packages=['subunit', 'subunit.tests'], |
3256 | package_dir={'subunit': 'python/subunit'}, |
3257 | scripts = [ |
3258 | + 'filters/subunit-1to2', |
3259 | + 'filters/subunit-2to1', |
3260 | 'filters/subunit2gtk', |
3261 | 'filters/subunit2junitxml', |
3262 | 'filters/subunit2pyunit', |