show summary at end of log file

Indeed the single "log" often isn't the easiest to consume output. But every individual test has its own <testname>-stdout and <testname>-stderr artifact, which is also available from http://autopkgtest.ubuntu.com ("artifacts.tar.gz" link). stderr and exit codes are also contained in the "summary" file (also in artifacts.tar.gz). Is that more suitable for you? If not, how would you propose to do that in a better way?

As autopkgtest can't interpret the contents of that there isn't that much more that we can do beyond that. I. e. the actual test output is just a blob from autopkgtest's view.

On Aug 14, 2015, at 03:19 PM, Martin Pitt wrote:

>Indeed the single "log" often isn't the easiest to consume output. But
>every individual test has its own <testname>-stdout and
><testname>-stderr artifact, which is also available from
>http://autopkgtest.ubuntu.com ("artifacts.tar.gz" link). stderr and exit
>codes are also contained in the "summary" file (also in
>artifacts.tar.gz). Is that more suitable for you? If not, how would you
>propose to do that in a better way?

Ah, good to know! Maybe instead of putting those only in the artifacts
tarball, they could be unpacked and linked from the autopkgtests.ubuntu.com
page?

>As autopkgtest can't interpret the contents of that there isn't that
>much more that we can do beyond that. I. e. the actual test output is
>just a blob from autopkgtest's view.

But the final PASS/FAIL results and exit codes are available to autopkgtest,
right? Maybe just a summary at the end could be printed with that
information?

Printing the summary at the end of logs indeed does sound like a good improvement, especially if a package has several tests and not the last one failed. I usually search for "- - results" in logs to jump between the test results, but this might be more obvious with a final summary.

Moving summary from artifacts.tar.gz to result.tar seems okay, but not sure if that's still needed once log.gz gets the summary too?

http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/commit/?id=773b00c

This bug was fixed in the package autopkgtest - 3.16.3

---------------
autopkgtest (3.16.3) unstable; urgency=medium

  * adt-setup-vm: Use /etc/cloud.cfg.d/ to avoid conffile prompts on upgrades.
    (LP: #1477626)
  * Don't invoke QEMU with -localtime. QEMU defaults to and Linux prefers the
    hw clock in UTC. This avoids large time jumps when doing NTP.
  * adt-setup-vm: Start autopkgtest shell after all other SysV init
    scripts/facilities, to avoid a too early start of tests while boot is
    still in progress.
  * doc/README.package-tests.rst: Use comma separator in examples.
  * VirtSubproc.check_exec(): Actually error out on stderr as documented.
  * adt-virt-ssh: Verify that sudo does not write errors, as that
    interferes with auxverbs and generally is a sign of host mis-configuration
    (typically wrong hostname).
  * ssh-setup-nova: Set hostname to "adt". This avoids overly long host names
    from defaulting to nova instance names (which can be very long).
  * ssh-setup/nova: Fix parsing of IP address from "nova show" to avoid
    tripping over instance names containing "network". (LP: #1481574)
  * Fix error message and code for "invalid test depends" errors.
  * tests/pep8: Ignore E402 ("module level import not at top of file"), as
    that's impossible to satisfy; we have to set sys.path before.
  * ssh-setup/nova: In cleanup(), wait until the instance gets deleted. This
    avoids name collisions and instance quota overflow.
  * adt-run: Always show summary at the end, which is particularly helpful for
    multiple tests. (LP: #1484991)
  * adt-run.1: Clarify that without any logging options adt-run only logs to
    stderr. (LP: #1485661)
  * adt-virt-lxc: If the executed command exits with 255, translate that to
    253 in the auxverb wrapper, as 255 is the exit code for failures of the
    auxverb itself. This avoids considering ordinary test failures as
    temporary testbed failures.
  * adt-run: Show host name, to more easily identify hosts with frequent
    failures.
  * adt-virt-lxc: Always call lxc-stop with --kill, as the containers are not
    precious and normal shutdown might hang.
  * adt-virt-lxc: In cleanup, stop LXC container before removing the shared
    downtmp, to ensure that we always clean up the running container. Also
    don't fail on errors. (LP: #1488879)
  * ssh-setup/nova: Add missing apt sources for -updates/-security
    restricted/multiverse pockets.

 -- Martin Pitt <email address hidden> Thu, 27 Aug 2015 19:54:51 +0200