io_uring: get rid of intermediate aux cqe caches
authorPavel Begunkov <asml.silence@gmail.com>
Mon, 18 Mar 2024 22:00:32 +0000 (22:00 +0000)
committerJens Axboe <axboe@kernel.dk>
Mon, 15 Apr 2024 14:10:24 +0000 (08:10 -0600)
commit902ce82c2aa130bea5e3feca2d4ae62781865da7
treeb1dc57d30ccaa7605ad5d675ce8e7a6c9756e7f8
parente5c12945be5016d681ff305ea7306fef5902219d
io_uring: get rid of intermediate aux cqe caches

io_post_aux_cqe(), which is used for multishot requests, delays
completions by putting CQEs into a temporary array for the purpose
completion lock/flush batching.

DEFER_TASKRUN doesn't need any locking, so for it we can put completions
directly into the CQ and defer post completion handling with a flag.
That leaves !DEFER_TASKRUN, which is not that interesting / hot for
multishot requests, so have conditional locking with deferred flush
for them.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Tested-by: Ming Lei <ming.lei@redhat.com>
Link: https://lore.kernel.org/r/b1d05a81fd27aaa2a07f9860af13059e7ad7a890.1710799188.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
include/linux/io_uring_types.h
io_uring/io_uring.c
io_uring/io_uring.h