Regressiontest failing

Well, this looks like ship algorithm error. I made some changes to this code some time ago, but we run the reggression tests and they went OK.
I will look on it...

Well, I tested suspicious revision (mine, about thips) - 7515 - OK.

Also another random revisions 7530 and 7550 are OK.

Perhaps rev. 7554 "Allow target quantity of 0 wares in economy. New workers wil..." is culprit???

Just for completeness, did YOU run the regression tests on current Trunk just to confirm and make sure it is not something on my machine?

Test is clean on my machine.

@Jens: Maybe one of your files got messed up, could you please get a fresh copy of trunk on your machine and run the tests again? It is comes up clean then, this bug is invalid.

Ok I did a clean bzr branch lp:widelands and it still fails... :-(

switched gcc from 4.8 to 4.9 - no change.

interesting detail... (sorry for the spam).

the line it mentions is:

   assert_equal(port1():get_wares("blackwood"), 1)

the error message is: expected '0' but was '1'

so... why does it expect 0 when the line says 1?

http://www.mroth.net/lunit/documentation-0.4.txt

        assert_equal( expected, actual, [msg] )

              Fails, if 'actual' is different from 'expected'. Make sure
              that you don't mix 'expected' and 'actual' because they are
              used to build a nice error message.

So that's that.

One thing you could try is to increase the time that the test is waiting for the ship to return the ware (line 24, sleep(8000)), to find out if that is the problem.

You could also run

bzr uncommit
bzr revert

and compile and test again, until you find the revision that is causing the problem for you.

I think I will try to go 100 revisions back to see what's there.

In the mean time I did some more testing: I loaded the savegame which is done immediately after the test, and the ship with the blackwood keeps standing there in the pond and does not go back to the first port to unload it as it is supposed to (from what I get from the test description).

@Jens
Yes, it fails here with current trunk (= rev. 7554)

I will look at fleet code that is responsible for picking a next destination of ship and sending the ship there...

But I have seen this failure before :)

Just wanted to tell you the same revision :-)

What exactly port1():get_wares("blackwood") returns? List, or count or true/false?

I think I have found a bug there, see merge request

According to https://wl.widelands.org/docs/wl/autogen_wl_map/#haswares, get_wares() returns a single integer in this case.

And as pointed out in #9, the order for assert_equal() is different from what is currently used. While you're on it, either change all or open a bug report.

so it returns count of blackwood - makes sense

I switched the values, I think this issue is present also in other test files though...

Just my 2c here: The regression test runs in full every time I build a Mac OS X binary for the nightlies, i.e. basically every work day. I do not see any failures there currently.

The reason the tests are flaky and fail on some machines is unfortunately due to the nature of Widelands coding.

I know one issue that is a definite problem: saving does not happen through the commandqueue, but through a side loaded channel that relies on realtime. That means that whenever a script issues a save request, saving will happen sometime soon after that. How much gametime passes depends on the busyness of the computer running the game and external factors - like FPS, delay on reading files and so on. Therefore all the tests contain so much sleep()s to make them less brittle.

Another issue that might be a problem, but I do not know: It could be that the simulation is not fully deterministic. That is just a more generalized version of the problem described in the last paragraph since the sideloading of save makes the game non-deterministic. What I mean here is that a game that runs at 100x for 1 minute might look different than the same game running at 1x for 100 minutes. I do not know if that is the case, ideally it should not be, but I am not sure if we ever made sure that it is/isn't the case.

I think here one of sleeps was very inconviently choosen (12000 ms), if ship managed to get to the port before, it returned failure, and if after, it reported success. But Im not 100% sure of this.

Generally, the problem are hardcoded times (sleeps) - sometimes, usually time of sailing is constant but times inbetween can vary a bit...

Green for me as well, thanks :-)

Fixed in build19-rc1.