ioengines.c:346: td_io_queue: Assertion `res == 0' failed

Assertion in ioengines.c::td_io_queue() fails for pthread_mutex_unlock()
on overlap_check mutex when serialize_overlap=1, io_submit_mode=offload,
and verify=<any> are used together.

backend.c::fio_io_sync() invokes td_io_queue(), which expects the caller
to have ownership of the overlap_check mutex when serialize_overlap
and offloading are configured, as part of the overlap-check interlock
with IO_U_F_FLIGHT. The mutex is not acquired for this path because it's
not an I/O requiring an overlap check.

The fix is to refine the conditional that triggers td_io_queue() to
release the overlap_check mutex. Rather than using broad config options,
the conditional now uses a new io_u flag named IO_U_F_OVERLAP_LOCK, which
is only set for the offload worker thread path that acquires the mutex.

Link: https://github.com/axboe/fio/issues/1520
Signed-off-by: Adam Horshack (horshack@live.com)

diff --git a/io_u.h b/io_u.h
index 206e24fee09ec955a05220203576a6e9c03220d3..50f2ad55421f4d39d760eabc86dde998dd75c374 100644 (file)
--- a/io_u.h
+++ b/io_u.h
@@ -21,6 +21,7 @@ enum {
        IO_U_F_TRIMMED          = 1 << 5,
        IO_U_F_BARRIER          = 1 << 6,
        IO_U_F_VER_LIST         = 1 << 7,
+       IO_U_F_OVERLAP_LOCK     = 1 << 8,
 };
 
 /*

diff --git a/ioengines.c b/ioengines.c
index e2316ee4e391d0d5899c960552fec4d19aa33ca2..843dd466373a28ae853b1b4e62c3cdbd6eeeff8c 100644 (file)
--- a/ioengines.c
+++ b/ioengines.c
@@ -341,9 +341,10 @@ enum fio_q_status td_io_queue(struct thread_data *td, struct io_u *io_u)
         * started the overlap check because the IO_U_F_FLIGHT
         * flag is now set
         */
-       if (td_offload_overlap(td)) {
+       if (io_u->flags & IO_U_F_OVERLAP_LOCK) {
                int res = pthread_mutex_unlock(&overlap_check);
                assert(res == 0);
+               io_u_clear(td, io_u, IO_U_F_OVERLAP_LOCK);
        }
 
        assert(fio_file_open(io_u->file));

diff --git a/rate-submit.c b/rate-submit.c
index 3cc17eaa560304b48fc2ec07f163b31fb204b9db..902a6c00b36c83486cc5bfd7b798f72c0dff1979 100644 (file)
--- a/rate-submit.c
+++ b/rate-submit.c
@@ -47,6 +47,7 @@ retry:
                assert(res == 0);
                goto retry;
        }
+       io_u_set(td, io_u, IO_U_F_OVERLAP_LOCK);
 }
 
 static int io_workqueue_fn(struct submit_worker *sw,