io_uring: optimise io_get_cqe()
authorPavel Begunkov <asml.silence@gmail.com>
Tue, 12 Apr 2022 14:09:51 +0000 (15:09 +0100)
committerJens Axboe <axboe@kernel.dk>
Sun, 17 Apr 2022 22:55:33 +0000 (16:55 -0600)
commitddf3eee6862f8a3e080a4e0b9c8401c47153b3ec
tree0e50e75eaae87c40d667e51af2da80628475b538
parent9a5303f1354ed77d29333581d0260f4fc3a784eb
io_uring: optimise io_get_cqe()

io_get_cqe() is expensive because of a bunch of loads, masking, etc.
However, most of the time we should have enough of entries in the CQ,
so we can cache two pointers representing a range of contiguous CQE
memory we can use. When the range is exhausted we'll go through a slower
path to set up a new range. When there are no CQEs avaliable, pointers
will naturally point to the same address.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/487eeef00f3146537b3d9c1a9cef2fc0b9a86f81.1649771823.git.asml.silence@gmail.com
[axboe: santinel -> sentinel]
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io_uring.c