io_uring: improve poll completion performance
authorJens Axboe <axboe@kernel.dk>
Thu, 19 Dec 2019 19:06:02 +0000 (12:06 -0700)
committerJens Axboe <axboe@kernel.dk>
Sat, 28 Dec 2019 23:06:24 +0000 (16:06 -0700)
commit31975737af7cde4480f0c3fce72be1426209ef5d
tree956f00e812cd84b6c5f503a8db2c70ac78b03b61
parenta47dabe63cce464bc1d8ff5286a477a36a6b9b81
io_uring: improve poll completion performance

For busy IORING_OP_POLL_ADD workloads, we can have enough contention
on the completion lock that we fail the inline completion path quite
often as we fail the trylock on that lock. Add a list for deferred
completions that we can use in that case. This helps reduce the number
of async offloads we have to do, as if we get multiple completions in
a row, we'll piggy back on to the poll_llist instead of having to queue
our own offload.

Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io_uring.c