I wonder about the placement of the 12 hour idle test.
Your assumption seems to be that the tester will idle the machine for > 12 hours before running checkbox, as evidinced by the idle check test and syslog check to look for errors. However, what if the tester idles for 12 hours then reboots the machine before starting checkbox? They wont necessarily know to not do that until the idle test itself runs after idle_check and syslog_check.
What about actually putting that at the end of the test run and making it all automatic:
at end of the run do a manual test for 12 hour IDLE that looks like this:
description:
PURPOSE:
To test that they system idles for 12 hours minimum
STEPS:
Select Test to begin the 12 hour idle test. Wait the prescribed amount of time and the
testing should complete automatically
VERIFICATION:
The verification of this test is automatic
command:
for x in `seq 1 12`; do echo "12 Hour Idle Test: Hour $x"; sleep 1h; done && True
followed up by idle_check and syslog_check just before finishing the run.
That, at least, guarantees that all the bits will work. My concern is that the way it's currently laid out is brittle (not sure if my idea is that much less brittle, but it seems so to me, feel free to poke holes)
I wonder about the placement of the 12 hour idle test.
Your assumption seems to be that the tester will idle the machine for > 12 hours before running checkbox, as evidinced by the idle check test and syslog check to look for errors. However, what if the tester idles for 12 hours then reboots the machine before starting checkbox? They wont necessarily know to not do that until the idle test itself runs after idle_check and syslog_check.
What about actually putting that at the end of the test run and making it all automatic:
at end of the run do a manual test for 12 hour IDLE that looks like this:
description:
PURPOSE:
To test that they system idles for 12 hours minimum
STEPS:
Select Test to begin the 12 hour idle test. Wait the prescribed amount of time and the
testing should complete automatically
VERIFICATION:
The verification of this test is automatic
command:
for x in `seq 1 12`; do echo "12 Hour Idle Test: Hour $x"; sleep 1h; done && True
followed up by idle_check and syslog_check just before finishing the run.
That, at least, guarantees that all the bits will work. My concern is that the way it's currently laid out is brittle (not sure if my idea is that much less brittle, but it seems so to me, feel free to poke holes)