test: improve evaluation of t0020.fio and t0021.fio
authorVincent Fu <vincent.fu@samsung.com>
Fri, 24 Mar 2023 14:16:14 +0000 (14:16 +0000)
committerVincent Fu <vincent.fu@samsung.com>
Thu, 13 Apr 2023 17:40:29 +0000 (13:40 -0400)
commit888dbd62c769448c4052f795f123f0d04a56fde3
tree0c11e6b69549ef7f7abdf6d554d4f9a33f4f5cd6
parentc994fa62425fbc1e5f62115c58829067bcc1e28f
test: improve evaluation of t0020.fio and t0021.fio

We previously used an arbirtrary test based on a count of consecutive
offsets to decide whether the offsets from t0020.fio and t0021.fio were
random or not. Replace this with a Wald-Wolfowitz runs test. This new
test calculates a statistic based on whether each offset is in the first
or second half of the LBA space. So it is not particularly powerful
although it has a more sound statistical justification than what we
originally had.

Recent changes to fio's default random number seeds produced false
positives with t0020.fio and prompted this change.

Consider calculating the test statistic directly instead of relying on
the statsmodel import in order to reduce the dependencies required.

References:
https://www.statology.org/runs-test-python/
https://www.geeksforgeeks.org/runs-test-of-randomness-in-python/
https://www.investopedia.com/terms/r/runs_test.asp
https://www.statsmodels.org/stable/generated/statsmodels.sandbox.stats.runs.runstest_1samp.html

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
ci/actions-install.sh
ci/appveyor-install.sh
t/run-fio-tests.py