ppa-dev-tools:add-results-output-to-tests-command

Last commit made on 2022-09-15
Get this branch:
git clone -b add-results-output-to-tests-command https://git.launchpad.net/ppa-dev-tools
Only Bryce Harrington can upload to this branch. If you are Bryce Harrington please log in for upload directions.

Branch merges

Branch information

Name:
add-results-output-to-tests-command
Repository:
lp:ppa-dev-tools

Recent commits

2cc3f55... by Bryce Harrington

subtest: Format subtests in table columns

769a703... by Bryce Harrington

subtest: Fix parsing to split on whitespace rather than ':'

When looking at real subtest data I noticed it was misparsing stuff
because the code expected a ':' separator after the subtest name.
However the results are just whitespace delimited.

Along with fixing that, add some real-world data to test cases for
things that would have broken with this logic.

259f65d... by Bryce Harrington

result: Fix incomplete listing of subtests when displaying results

The status() command looks for the first FAIL returned from
get_subtests(), however since the latter routine is implemented as an
iterator, if one of the middle subtests fails then the next caller to
get_subtests() would start from where the previous call stopped.

This caused a failure when processing this set of subtests:

    autopkgtest [15:01:22]: @@@@@@@@@@@@@@@@@@@@ summary
    light-simple PASS
    branding FAIL non-zero exit status 139
    ec-x25519 PASS
    core-simple PASS
    full-simple PASS
    extras-simple PASS
    core-module-deps PASS
    full-module-deps PASS
    light-module-deps PASS
    extras-module-deps PASS

The reporting code first looked for the overall status via the
result.status property. That scanned down the list to the 'branding'
test to determine that the autopkgtest run failed. Next, the reporting
code prints the subtests using result.get_subtests(), however that
call's next yield was the ex-x25519 test. This resulted in the output
only displaying results from ec-x25519 down.

20d8854... by Bryce Harrington

subtest: Fix handling of subtests failing with exit status

Test results may look like:

    autopkgtest [15:01:22]: @@@@@@@@@@@@@@@@@@@@ summary
    light-simple PASS
    branding FAIL non-zero exit status 139
    ec-x25519 PASS
    core-simple PASS
    full-simple PASS

Since there is descriptive text after the FAIL, we can't simply look at
the last word on the line but must search for PASS/FAIL/etc. at any
place in the string.

a95264b... by Bryce Harrington

ppa: Don't display subtests for successful runs

lp-test-ppa shows detailed information for failures but condenses
successful runs to a single line. This makes debugging a touch easier
when there's problems, while ensuring the results are easy to copy and
paste into reports once everything succeeds.

815f920... by Bryce Harrington

Lookup current development release instead of hardcoding

7ac5c67... by Bryce Harrington

ppa: Add Results to the tests command

Running a command like this now prints out (copious) results
information:

    ppa tests ppa:bryce/nginx-merge-v1.22.0-1

6e1a33c... by Bryce Harrington

ppa: Use EX_OK exit code for success

0044bc3... by Bryce Harrington

ppa: Exit with os.EX_OK on success

508eed2... by Bryce Harrington

ppa: Show usage for missing parameters

In practice, argparse already catches missing parameter issues, so this
is theoretical. However, the intent of these checks is to assure proper
inputs were passed, and displaying usage would be the logical result if
somehow they were missing.