io_uring/rsrc: use non-pcpu refcounts for nodes
authorPavel Begunkov <asml.silence@gmail.com>
Tue, 4 Apr 2023 12:39:45 +0000 (13:39 +0100)
committerJens Axboe <axboe@kernel.dk>
Tue, 4 Apr 2023 15:30:39 +0000 (09:30 -0600)
commitb8fb5b4fdd67f9d18109c5d21d44a8bd4ddb608b
treed88a6c8049aeb16442461afa53182a2c603a5eb6
parente3ef728ff07b42668e7e12f49cd2f9055e064ec1
io_uring/rsrc: use non-pcpu refcounts for nodes

One problem with the current rsrc infra is that often updates will
generates lots of rsrc nodes, each carry pcpu refs. That takes quite a
lot of memory, especially if there is a stall, and takes lots of CPU
cycles. Only pcpu allocations takes >50 of CPU with a naive benchmark
updating files in a loop.

Replace pcpu refs with normal refcounting. There is already a hot path
avoiding atomics / refs, but following patches will further improve it.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/e9ed8a9457b331a26555ff9443afc64cdaab7247.1680576071.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring/rsrc.c
io_uring/rsrc.h