io_uring: save atomic dec for inline executed reqs
authorPavel Begunkov <asml.silence@gmail.com>
Tue, 19 Jan 2021 13:32:47 +0000 (13:32 +0000)
committerJens Axboe <axboe@kernel.dk>
Mon, 25 Jan 2021 15:18:37 +0000 (08:18 -0700)
commitac4e5af860a8e1c2f4d73e9a363981b7d1228ca5
tree80c04a656353a2245ec3071ebb8303ccd8e398cd
parent3fe8e8cfc997cbb9aa1cb98d7beab2af05025d2d
io_uring: save atomic dec for inline executed reqs

When a request is completed with comp_state, its completion reference
put is deferred to io_submit_flush_completions(), but the submission
is put not far from there, so do it together to save one atomic dec per
request. That targets requests that complete inline, e.g. buffered rw,
send/recv.

Proper benchmarking haven't been conducted but for nops(batch=32) it was
around 7901 vs 8117 KIOPS (~2.7%), or ~4% per perf profiling.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io_uring.c