verify: disable write sequence checks with norandommap and iodepth > 1

With norandommap for async I/O engines specifying I/O depth > 1, it is
possible that two or more writes with the same offset are queued at once.
When fio tries to verify the block, it may find a numberio mismatch
because the writes did not land on the media in the order that they were
queued. Avoid these spurious failures by disabling sequence number
checking. Users will still be able to enable sequence number checking
if they explicitly set the verify_header_sequence option.

fio -name=verify -ioengine=libaio -rw=randwrite -verify=sha512 -direct=1 \
-iodepth=32 -filesize=16M -bs=512 -norandommap=1 -debug=io,verify

Below is the truncated log for the above command demonstrating the issue.
This includes extra log entries when write sequence number is saved and
retrieved.

set: io_u->numberio=28489, off=0x5f2400
queue: io_u 0x5b8039e30d40: off=0x5f2400,len=0x200,ddir=1,file=verify.0.0
set: io_u->numberio=28574, off=0x5f2400
iolog: overlap 6235136/512, 6235136/512
queue: io_u 0x5b8039e75500: off=0x5f2400,len=0x200,ddir=1,file=verify.0.0
complete: io_u 0x5b8039e75500: off=0x5f2400,len=0x200,ddir=1,file=verify.0.0
complete: io_u 0x5b8039e30d40: off=0x5f2400,len=0x200,ddir=1,file=verify.0.0

retrieve: io_u->numberio=28574, off=0x5f2400
queue: io_u 0x5b8039e1db40: off=0x5f2400,len=0x200,ddir=0,file=verify.0.0

bad header numberio 28489, wanted 28574 at file verify.0.0 offset 6235136, length 512 (requested block: offset=6235136, length=512)

Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>

diff --git a/HOWTO.rst b/HOWTO.rst
index 473adc4bb2c9205e3cdc843a79f5d1ff472880fd..41ebabee3c438ca40dbae2df4e8db31463ba66e1 100644 (file)
--- a/HOWTO.rst
+++ b/HOWTO.rst
@@ -1568,9 +1568,8 @@ I/O type
        disabled. If this option is used with :option:`verify` and multiple blocksizes
        (via :option:`bsrange`), only intact blocks are verified, i.e.,
        partially-overwritten blocks are ignored. With an async I/O engine and an I/O
-       depth > 1, it is possible for the same block to be overwritten, which can
-       cause verification errors. Either do not use norandommap in this case, or also
-       use the lfsr random generator.
+       depth > 1, header write sequence number verification will be disabled. See
+       :option:`verify_write_sequence`.
 
 .. option:: softrandommap=bool
 

diff --git a/fio.1 b/fio.1
index 277075ed8539e295443f3d7a562eda335380d561..d7485bb4930d4b921e4e7053eba4d1f83b36693c 100644 (file)
--- a/fio.1
+++ b/fio.1
@@ -1371,9 +1371,8 @@ and that some blocks may be read/written more than once. If this option is
 used with \fBverify\fR then \fBverify_header_seed\fR will be disabled. If this
 option is used with \fBverify\fR and multiple blocksizes (via \fBbsrange\fR),
 only intact blocks are verified, i.e., partially-overwritten blocks are
-ignored.  With an async I/O engine and an I/O depth > 1, it is possible for
-the same block to be overwritten, which can cause verification errors.  Either
-do not use norandommap in this case, or also use the lfsr random generator.
+ignored. With an async I/O engine and an I/O depth > 1, header write sequence
+number verification will be disabled. See \fBverify_write_sequence\fR.
 .TP
 .BI softrandommap \fR=\fPbool
 See \fBnorandommap\fR. If fio runs with the random block map enabled and

diff --git a/init.c b/init.c
index 91f056e36a7eb3285294db07f286d01335dcaf30..00a3a8c7e46ff6566ea997f34d2188aded2c750e 100644 (file)
--- a/init.c
+++ b/init.c
@@ -862,6 +862,17 @@ static int fixup_options(struct thread_data *td)
                                o->verify_header_seed = 0;
                }
 
+               if (o->norandommap && !td_ioengine_flagged(td, FIO_SYNCIO) &&
+                   o->iodepth > 1) {
+                       /*
+                        * Disable write sequence checks with norandommap and
+                        * iodepth > 1.
+                        * Unless we were explicitly asked to enable it.
+                        */
+                       if (!fio_option_is_set(o, verify_write_sequence))
+                               o->verify_write_sequence = 0;
+               }
+
                /*
                 * Disable rand_seed check when we have verify_backlog,
                 * zone reset frequency for zonemode=zbd, or norandommap.