Howdy,

On Sun, Nov 30, 2008 at 4:46 PM, Brian Granger <email address hidden> wrote:
> Ville,
>
>> I didn't think it matters - the test case does output to stdout by
>> design, since it tests interactive stuff (autocall conversion). I
>> don't think it makes sense to redirect stdout here, since nose already
>> does it.
>
> What test is failing, I want to have a look.
>
> I don't think we want to get into the habit of allowing tests to write
> to stdout...Ideally we will be adding more and more tests of the
> interactive stuff, and if more of them do this, our test suite will be
> a huge mess when it gets run.
>
> We now actually have quite a few tests and it is really important to
> leave no doubts in folks mind what tests are passing and which are
> failing.  When I see random noise (that doesn't even say what test it
> is associated with), things *look* broken, even if there are not.  If
> we write more tests that have noise on stdout, it will be very
> difficult to sort out what is going on.  So, I am still not conviced
> we want to allow this type of thing.
>
> But, I would like to have a look at this particular test, so if you
> could points me in the right direction, that would be great.
>
>> Obviously, it can be done, especially since it's the only test that
>> does it so far - but yet it's also among the first tests that
>> excercises interactive stuff, and I don't think we should set a
>> precedent about not allowing stdout for tests.
>
> I would like to know Fernando's thoughts on this - maybe there is a
> way that we can have nose handle this stuff automatically.  I would be
> fine with that.  Fernando would know if this is possible and could
> help us implement it.

What's strange about those particular tests is that somehow they are
getting to stdout even after nose's capturing.   I have plenty of
tests already in my branch that do interactive things but you only see
their output if you run nose with -s, while in this branch there's
still some output.  Consider:

- Trunk:
bic128[~]> iptest
----------------------------------------------------------------------
Ran 420 tests in 29.305s

OK (SKIP=4)

- Ville's branch:
bic128[~]> iptest
>f2("a b c")
>f1("a", "b", "c")
>f1(1,2,3)
>f2(4)
 os.path.isdir(path)
----------------------------------------------------------------------
Ran 434 tests in 31.652s

OK (SKIP=4)


I fully agree with Brian that, at least when run *without* -s, the
test suite should be silent.  Scipy got into the habit of having a
hideously noisy test suite from the start (years ago) and now people
are going through a lot of pain trying to clean it up. Let's not start
down the wrong path ourselves.

There's another problem: in this branch, '-v' or '-s' currently don't
work for iptest:

bic128[~]> iptest -v
WARNING:
Error in Arguments: "Illegal option '-v'"

bic128[~]> iptest -s
[... snip ipython help screen... ]
WARNING:
Error in Arguments: "Ambiguous option '-s';  matches ['screen_length',
'sl', 'separate_in', 'si', 'separate_out', 'so', 'separate_out2',
'so2', 'system_verbose']"


Since this works in trunk, it's likely just missing merges from trunk
on this branch.

I'll continue looking at this branch until we get it in shape for
commit, it's excellent that more tests are going in!

Cheers,

f