io_uring: lockless task list
authorDylan Yudaken <dylany@fb.com>
Wed, 22 Jun 2022 13:40:23 +0000 (06:40 -0700)
committerJens Axboe <axboe@kernel.dk>
Mon, 25 Jul 2022 00:39:15 +0000 (18:39 -0600)
commitf88262e60bb9cb5740891672ce9f405e7f9393e5
treebca01661645c35ef7958fd5b0a5f9438d9b9b477
parentc34398a8c018e0d3d2d30b718d03c7290c696f51
io_uring: lockless task list

With networking use cases we see contention on the spinlock used to
protect the task_list when multiple threads try and add completions at once.
Instead we can use a lockless list, and assume that the first caller to
add to the list is responsible for kicking off task work.

Signed-off-by: Dylan Yudaken <dylany@fb.com>
Link: https://lore.kernel.org/r/20220622134028.2013417-4-dylany@fb.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
include/linux/io_uring_types.h
io_uring/io_uring.c
io_uring/tctx.c
io_uring/tctx.h