io_uring/rsrc: move node refs == 0 checking outside of hot path
One thing that is slightly suboptimal on the current resource node
handling, is that putting a node neccesitates decrementing the
ref count AND checking if it hit zero. While this isn't an atomic
operation (as we're under the ctx lock), it does mean the decrement
cannot be pipelined.
Move the dead node checking to unregistration time. The holder of a
node will still decrement the reference count, but it need not check
if it hit zero. It will, by definition, never hit zero until it gets
unregistered, as the table itself holds a reference.
At unregistration time, if the node ref count isn't zero when the
table reference is dropped, then its known that someone else holds a
reference to it and will put it at some point. For this situation,
store an entry for later reaping in ctx->rsrc_free. Whenever a batch
of nodes is flushed, whether ctx->rsrc_free is empty or not is
checked, and previously released nodes will be freed.
With that, the overhead of putting a node from the hot path is reduced
quite a bit, as everything pipelines nicely and doesn't stall on
bringing in the node cacheline.
Signed-off-by: Jens Axboe <axboe@kernel.dk>