Code review comment for lp:~clint-fewbar/charm-tools/charm-tests-spec

Revision history for this message
Martin Pool (mbp) wrote :

> This seems very reasonable.

Yes, thanks for working on this.

> Some nitpicky comments:
>
> It seems that a non-zero exit code indicates a failure, and a zero exit code
> indicates success? I guess this is fairly standard, but it would be nice to
> have it documented explicitly. Is it necessary to use 1 for failure (and
> perhaps anything else for errors), or is any non-zero exit code sufficient?

I think it would be well worth while making this explicit from day one: most test frameworks eventually need to distinguish "test failed" from "test couldn't be run", "results were inconclusive", "test timed out", etc. If you specify that a failure must be, say, 1, then you can add other things later if you want.

« Back to merge proposal