Merge lp:~smerrill/pressflow/simpletest-xml-junit into lp:pressflow

Proposed by Steven Merrill on 2009-10-12
Status: Merged
Merged at revision: not available
Proposed branch: lp:~smerrill/pressflow/simpletest-xml-junit
Merge into: lp:pressflow
Diff against target: 127 lines 1 file modified
To merge this branch: bzr merge lp:~smerrill/pressflow/simpletest-xml-junit
Reviewer Review Type Date Requested Status
David Strauss 2009-10-12 Approve on 2009-10-15
Josh Koenig 2009-10-12 Approve on 2009-10-14
Review via email: mp+13224@code.launchpad.net
To post a comment you must log in.
Steven Merrill (smerrill) wrote :

This is an implementation of d.o #602332, which extends the work done on http://www.computerminds.co.uk/hudson-and-simpletest and brings it up to sync with Pressflow r49+ / SimpleTest 6.x-2.9+.

Steven Merrill (smerrill) wrote :

As an example, this is our after build script in Hudson that leverages this script to output XML test results into the scripts/tests directory:

/usr/bin/php scripts/run-tests.sh --url http://example.com --xml scripts/tests --all

David Strauss (davidstrauss) wrote :

I'm going to run this past the folks working on Hudson at The Economist (which includes me). We're certainly using jUnit XML results with Pressflow, and I'll see how this proposed merge differs from what we're using.

David Strauss (davidstrauss) wrote :

One note, though. This needs to stay in the Pressflow version because Pressflow uses the D7 CLI detection method:
simpletest_script_init(NULL);

Josh Koenig (joshkoenig) wrote :

This looks quite good to me, and is actually somewhat better than the legacy script we are using now.

I like the notion of reading test results from the database after they're complete rather than building in real-time.

I also like using DomDocument rather than SimpleXML as I've seen SimpleXML choke in some cases by being unreasonably strict.

I approve of this merge. I would also like to get Jimmy (boombatower) to think about including something like this in D7, but am happy if Pressflow leads on this. :)

review: Approve
Josh Koenig (joshkoenig) wrote :

Ah, I see it's on d.o already. Duh. +1'ed it there also.

Nice Work!

> This looks quite good to me, and is actually somewhat better than the legacy
> script we are using now.
>
> I like the notion of reading test results from the database after they're
> complete rather than building in real-time.
>
> I also like using DomDocument rather than SimpleXML as I've seen SimpleXML
> choke in some cases by being unreasonably strict.
>
> I approve of this merge. I would also like to get Jimmy (boombatower) to think
> about including something like this in D7, but am happy if Pressflow leads on
> this. :)

60. By Steven Merrill on 2009-10-15

Code style cleanups.

61. By Steven Merrill on 2009-10-15

A few more small style tweaks.

62. By Steven Merrill on 2009-10-15

Change to use Pressflow/D7-style cli detection.

Steven Merrill (smerrill) wrote :

I've updated this branch to use the D7-style CLI detection and cleaned up the code style a bit, including removing the commented-out code.

I just double-checked it on a full Hudson-initiated run of all of our SimpleTests and it's working fine.

David Strauss (davidstrauss) wrote :

Merging now...

review: Approve

Preview Diff

1=== modified file 'scripts/run-tests.sh'
2--- scripts/run-tests.sh 2009-09-20 21:01:42 +0000
3+++ scripts/run-tests.sh 2009-10-15 00:29:09 +0000
4@@ -98,8 +98,16 @@
5 list($last_prefix, $last_test_class) = simpletest_last_test_get($test_id);
6 simpletest_log_read($test_id, $last_prefix, $last_test_class);
7
8-// Display results before database is cleared.
9-simpletest_script_reporter_display_results();
10+if ($args['xml']) {
11+ echo $args['xml'];
12+ // Save results as xml before database is cleared.
13+ simpletest_script_reporter_xml_results();
14+
15+}
16+else {
17+ // Display results before database is cleared.
18+ simpletest_script_reporter_display_results();
19+}
20
21 // Cleanup our test results.
22 simpletest_clean_results_table($test_id);
23@@ -149,6 +157,9 @@
24
25 --verbose Output detailed assertion messages in addition to summary.
26
27+ --xml Output verbose test results to a specified directory using the JUnit
28+ test reporting format. Useful for integrating with Hudson.
29+
30 <test1>[,<test2>[,<test3> ...]]
31
32 One or more tests to be run. By default, these are interpreted
33@@ -194,7 +205,8 @@
34 'test_names' => array(),
35 // Used internally.
36 'test-id' => NULL,
37- 'execute-batch' => FALSE
38+ 'execute-batch' => FALSE,
39+ 'xml' => '',
40 );
41
42 // Override with set values.
43@@ -274,7 +286,7 @@
44 if (!empty($args['url'])) {
45 $parsed_url = parse_url($args['url']);
46 $host = $parsed_url['host'] . (isset($parsed_url['port']) ? ':' . $parsed_url['port'] : '');
47- $path = $parsed_url['path'];
48+ $path = isset($parsed_url['path']) ? $parsed_url['path'] : '';
49 }
50
51 $_SERVER['HTTP_HOST'] = $host;
52@@ -482,6 +494,75 @@
53 echo "\n";
54 }
55
56+/*
57+ * Display test results.
58+ */
59+function simpletest_script_reporter_xml_results() {
60+ global $args, $test_id, $results_map;
61+
62+ echo "\n";
63+ $end = timer_stop('run-tests');
64+ echo "XML Test run duration: " . format_interval($end['time'] / 1000);
65+ echo "\n";
66+
67+ // Report results.
68+ echo "Detailed test results:\n";
69+ echo "----------------------\n";
70+ echo "\n";
71+
72+ $results_map = array(
73+ 'pass' => 'Pass',
74+ 'fail' => 'Fail',
75+ 'exception' => 'Exception',
76+ );
77+
78+ $results = db_query("SELECT * FROM {simpletest} WHERE test_id = %d ORDER BY test_class, message_id", $test_id);
79+
80+ $test_class = '';
81+ $xml_files = array();
82+
83+ while ($result = db_fetch_object($results)) {
84+ if (isset($results_map[$result->status])) {
85+ if ($result->test_class != $test_class) {
86+ // Display test class every time results are for new test class.
87+ if (isset($xml_files[$test_class])) {
88+ file_put_contents($args['xml'] . '/' . $test_class . '.xml', $xml_files[$test_class]['doc']->saveXML());
89+ unset($xml_files[$test_class]);
90+ }
91+ $test_class = $result->test_class;
92+ if (!isset($xml_files[$test_class])) {
93+ $doc = new DomDocument('1.0');
94+ $root = $doc->createElement('testsuite');
95+ $root = $doc->appendChild($root);
96+ $xml_files[$test_class] = array('doc' => $doc, 'suite' => $root);
97+ }
98+ }
99+ // Save the result into the XML:
100+ $case = $xml_files[$test_class]['doc']->createElement('testcase');
101+ $case->setAttribute('classname', $test_class);
102+ list($class, $name) = explode('->', $result->function, 2);
103+ $case->setAttribute('name', $name);
104+
105+ if ($result->status == 'fail') {
106+ $fail = $xml_files[$test_class]['doc']->createElement('failure');
107+ $fail->setAttribute('type', 'failure');
108+ $fail->setAttribute('message', $result->message_group);
109+ $text = $xml_files[$test_class]['doc']->createTextNode($result->message);
110+ $fail->appendChild($text);
111+ $case->appendChild($fail);
112+ }
113+
114+ $xml_files[$test_class]['suite']->appendChild($case);
115+ }
116+ }
117+
118+ // Save the last one:
119+ if (isset($xml_files[$test_class])) {
120+ file_put_contents($args['xml'] . '/' . $test_class . '.xml', $xml_files[$test_class]['doc']->saveXML());
121+ unset($xml_files[$test_class]);
122+ }
123+}
124+
125 /**
126 * Display test results.
127 */

Subscribers

People subscribed via source and target branches

to status/vote changes: