t/steadystate_tests: relax acceptance criterion

Relax the acceptance criterion for tests that do not attain steady
state. Originally, only a 10ms difference between expected and actual
runtime was allowed. Change this to 50ms. I observed a spurious failure
where a test ran for 10017ms instead of the expected 10000ms.

Signed-off-by: Vincent Fu <vincent.fu@wdc.com>

diff --git a/t/steadystate_tests.py b/t/steadystate_tests.py
index 9122a60fda052b6352bc475f289e154ee3cf6541..b55a67ac1cccdd23f6981670947785e6511dbbb5 100755 (executable)
--- a/t/steadystate_tests.py
+++ b/t/steadystate_tests.py
@@ -187,7 +187,7 @@ if __name__ == '__main__':
                     # check runtime, confirm criterion calculation, and confirm that criterion was not met
                     expected = job['timeout'] * 1000
                     actual = jsonjob['read']['runtime']
-                    if abs(expected - actual) > 10:
+                    if abs(expected - actual) > 50:
                         line = 'FAILED ' + line + ' ss not attained, expected runtime {0} != actual runtime {1}'.format(expected, actual)
                     else:
                         line = line + ' ss not attained, runtime {0} != ss_dur {1} + ss_ramp {2},'.format(actual, job['ss_dur'], job['ss_ramp'])
@@ -215,12 +215,12 @@ if __name__ == '__main__':
             else:
                 expected = job['timeout'] * 1000
                 actual = jsonjob['read']['runtime']
-                if abs(expected - actual) < 10:
-                    result = 'PASSED '
-                    passed = passed + 1
-                else:
+                if abs(expected - actual) > 50:
                     result = 'FAILED '
                     failed = failed + 1
+                else:
+                    result = 'PASSED '
+                    passed = passed + 1
                 line = result + line + ' no ss, expected runtime {0} ~= actual runtime {1}'.format(expected, actual)
             print(line)
             if 'steadystate' in jsonjob: