Comment 2 for bug 587886

For Graham's case the broken test clearly appears in the testrunner log file, but the entire suite still reported SUCCESS:

test: xx-temporary-blob-storage_txt
time: 2010-05-28 12:26:36.103693Z
failure: xx-temporary-blob-storage_txt [ multipart
Content-Type: text/x-traceback;charset=utf8,language=python
traceback
1AD
Failed doctest test for xx-temporary-blob-storage.txt
  File "lib/canonical/launchpad/pagetests/webservice/xx-temporary-blob-storage.txt", line 0

----------------------------------------------------------------------
File "lib/canonical/launchpad/pagetests/webservice/xx-temporary-blob-storage.txt", line 77, in xx-temporary-blob-storage.txt
Failed example:
    job.metadata
Differences (ndiff with -expected +actual):
    + {}
0
]

It appears that whatever ran the doctests failed to communicate that failure to its parent process using an exit code, or whatever ran the doctests may have not noticed the failure in that individual test. The ec2test.remote.SummaryResult object did correctly parse and report the error when it saw it in the output stream, but what the stream parser notices does not affect the test harness' overall outcome.