iomap: support IOCB_DIO_CALLER_COMP
authorJens Axboe <axboe@kernel.dk>
Sat, 8 Jul 2023 16:01:50 +0000 (10:01 -0600)
committerJens Axboe <axboe@kernel.dk>
Fri, 21 Jul 2023 16:34:42 +0000 (10:34 -0600)
commit20e869e469459a5653aa45245e10d4f279546a4b
tree175376031b0154596457b7d666e66a3fcd3cabb6
parent6f7b3f2ab4e273a1557080bf0932b0e464c89cb2
iomap: support IOCB_DIO_CALLER_COMP

If IOCB_DIO_CALLER_COMP is set, utilize that to set kiocb->dio_complete
handler and data for that callback. Rather than punt the completion to a
workqueue, we pass back the handler and data to the issuer and will get
a callback from a safe task context.

Using the following fio job to randomly dio write 4k blocks at
queue depths of 1..16:

fio --name=dio-write --filename=/data1/file --time_based=1 \
--runtime=10 --bs=4096 --rw=randwrite --norandommap --buffered=0 \
--cpus_allowed=4 --ioengine=io_uring --iodepth=$depth

shows the following results before and after this patch:

Stock Patched Diff
=======================================
QD1 155K 162K + 4.5%
QD2 290K 313K + 7.9%
QD4 533K 597K +12.0%
QD8 604K 827K +36.9%
QD16 615K 845K +37.4%

which shows nice wins all around. If we factored in per-IOP efficiency,
the wins look even nicer. This becomes apparent as queue depth rises,
as the offloaded workqueue completions runs out of steam.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/iomap/direct-io.c