autopkgtest fail in version 14.2 on test-display-streaming

expected test file from git show upstream/master:server/tests/base_test.ppm

mismatching output created by the tests

Actually I realuzed that the old test 0.14.0-1ubuntu5 is working and also had no regression-test.py nor the ppm files.
But the code in server/tests/test-display-base.c calls that python helper just the same:
  execve("regression-test.py", argv, envp);
So why isn't that failing, maybe it was never a valid test to begin with?.

I entered a test env of the test on Disco (with these issues present) and checked how/why the test complete there.

It has an RC of 0, but that it has always if not crashing.
Still no files around:
ubuntu@autopkgtest:/tmp/autopkgtest.xBVfjt/build.efW/src$ find -name '*.ppm*'
ubuntu@autopkgtest:/tmp/autopkgtest.xBVfjt/build.efW/src$ find -name '*regression*'

Function regression_test is hit often.
Thread 1 "test-display-st" hit Breakpoint 1, regression_test () at test-display-base.c:101
101 {

There a child is supposed to start the test:
 114 pid = fork();
 115 if (pid == 0) {
 116 char buf[PATH_MAX];
 117 char *argv[] = { NULL };
 118 char *envp[] = {buf, NULL};
 119
 120 snprintf(buf, sizeof(buf), "PATH=%s", getenv("PATH"));
 121 execve("regression-test.py", argv, envp);

Setting gdb to follow that
(gdb) set follow-fork-mode child
(gdb) set detach-on-fork off
(gdb) run --automated-tests

And yeah it now is confirmed that the regression test always only "ran" the test (which if breaking on execution would flag a bug) but it never got to compare the actual output in the former modes.

So my upload will fix that to be on the same level as it was before - but we need upstream and some understanding if we can make it actually work (create valid ppm files with output that matches).

There wasn't a spice project - but since spice-* is under one upstream umbrella spice-vdi is just as fine - important is that the gitlab tracker works.

This bug was fixed in the package spice - 0.14.2-0ubuntu2

---------------
spice (0.14.2-0ubuntu2) eoan; urgency=medium

  * Fixup autpkgtest (LP: #1834286)
    These changes will make the test able to run again, but not output mismatch
    errors (this matches the behavior before 0.14.2). Upstream discussion
    started on how to resolve that as a next step, more details at the LP bug.
    - d/t/automated-tests: spice-common moved into dir subprojects
    - d/t/automated-tests: option --enable-automated-tests now is always on"
    - d/t/automated-tests, d/t/control: make tests more debuggable by allowing
      stderr
    - d/t/control: install new test dependency python-pil
    - d/t/base_test.ppm, d/t/regression-test.py: provide test resources from
      upstream git not part of the released tarball anymore
    - d/source/include-binaries: allow binary base_test.ppm in package

 -- Christian Ehrhardt <email address hidden> Tue, 25 Jun 2019 12:59:01 +0200