io_uring/rw: add write support for IOCB_DIO_CALLER_COMP
authorJens Axboe <axboe@kernel.dk>
Sat, 8 Jul 2023 16:03:52 +0000 (10:03 -0600)
committerJens Axboe <axboe@kernel.dk>
Fri, 21 Jul 2023 16:34:34 +0000 (10:34 -0600)
commit6f7b3f2ab4e273a1557080bf0932b0e464c89cb2
tree97af309f8bb9b65ea58f28a3f18efa38d463e191
parente29b3962d5974243c43a19c6877ced47f8845a35
io_uring/rw: add write support for IOCB_DIO_CALLER_COMP

If the filesystem dio handler understands IOCB_DIO_CALLER_COMP, we'll
get a kiocb->ki_complete() callback with kiocb->dio_complete set. In
that case, rather than complete the IO directly through task_work, queue
up an intermediate task_work handler that first processes this callback
and then immediately completes the request.

For XFS, this avoids a punt through a workqueue, which is a lot less
efficient and adds latency to lower queue depth (or sync) O_DIRECT
writes.

Only do this for non-polled IO, as polled IO doesn't need this kind
of deferral as it always completes within the task itself. This then
avoids a check for deferral in the polled IO completion handler.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring/rw.c