Code review comment for lp:~javier.collado/checkbox/bug990075-2

I think I should let my feelings known about this functionality. As I've said before, I think the whitelist should be authoritative. The jobs should run in strict whitelist order and if any cannot be run due to failing dependencies than this should be reflected in the test report. For example, you could have job output like:

suspend/wireless_after_suspend Unitiated Did not run because suspend/suspend_advanced_auto failed

network/ping Unitiated Did not run because network/detect has not been run yet

I appreciate that people may be frustrated if they run tests and they see these messages if they have not been given a chance to rectify them. But two things to consider. One is that giving feedback *in Checkbox* does not help in the case of fully automated runs (as we do for server testing and SRU), the same result will occur. The other is that we could create a simple tool which validates a whitelist and points out any issues without having to run Checkbox itself. Take the precedent of a compiler or interpreter - it does not try to fix your program for you on the fly! The whitelists should be the same - if it's not right then A:) your results aren't right B:) you know why

I can't envision how we will have people coming to us saying they don't understand why their tests weren't run properly if we follow a simple scheme such as this.

Please, let me know what you think.