requirements job failing on c-c-s causing submission-resources to appear as a skipped test

requirements allows you to insert arbitrary links (name and URL) in the xml report. It was placed in miscellanea/submission-resources to avoid maintaining resources "the old way", with the downside that, as you observed, if the files it depends on are not present, it will barf and cause inconsistent results. This is mostly cosmetic because all the other dependencies of submission-resources will run; and that's the whole point, other than that, we don't really care about submission-resources itself.

For correctness and to avoid confusion, requirements resource should NOT exit with failure if the file it tries to read doesn't exist.

The command for the requirements job is:

command: [ -f $PLAINBOX_SESSION_SHARE/requirements_docs.txt ] && cat $PLAINBOX_SESSION_SHARE/requirements_docs.txt

This can change to:

command: if [ -f $PLAINBOX_SESSION_SHARE/requirements_docs.txt ]; then cat $PLAINBOX_SESSION_SHARE/requirements_docs.txt; else true; fi

Using if/then/else/fi is needed because just adding ||true at the end of the original command will not fail if the file *exists* but is unreadable (if cat blahblah returns error).

requirements doesn't exist in checkbox, so marking as invalid for that project.

I've seen this also in default.whitelist, although it does not really failed the submission now as @roadmr mentioned,
prior to the requirement job, would be nice to have another sample job to create example requirements_docs.txt.
Based on description in requirements job, i'm not sure the exactly format it should be.

We can improve the description if you think it'd be easier.

The description says:

Each requirement should have two keys with their respective
 values:
 name: (to be used as the anchor text)
 link: (the actual URL)

So a valid requirements file would be:

name: Yung's home page
link: http://yung.com

name: Sam's home page
link: http://phlin.com