io_uring: optimise task_work submit flushing
authorPavel Begunkov <asml.silence@gmail.com>
Thu, 17 Jun 2021 17:14:07 +0000 (18:14 +0100)
committerJens Axboe <axboe@kernel.dk>
Fri, 18 Jun 2021 15:22:02 +0000 (09:22 -0600)
commitebd0df2e63426bbd9ed50966e888c87eac88fc30
tree816227ee88250df69422c7f1b0006f404d6b6fc0
parent3f18407dc6f2db0968daaa36c39a772c2c9f8ea7
io_uring: optimise task_work submit flushing

tctx_task_work() tries to fetch a next batch of requests, but before it
would flush completions from the previous batch that may be sub-optimal.
E.g. io_req_task_queue() executes a head of the link where all the
linked may be enqueued through the same io_req_task_queue(). And there
are more cases for that.

Do the flushing at the end, so it can cache completions of several waves
of a single tctx_task_work(), and do the flush at the very end.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/3cac83934e4fbce520ff8025c3524398b3ae0270.1623949695.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io_uring.c