io_uring: mimimise io_cqring_wait_schedule
authorPavel Begunkov <asml.silence@gmail.com>
Thu, 5 Jan 2023 11:22:25 +0000 (11:22 +0000)
committerJens Axboe <axboe@kernel.dk>
Sun, 29 Jan 2023 22:17:40 +0000 (15:17 -0700)
commit846072f16eed3b3fb4e59b677f3ed8afb8509b89
tree8e0d28c8f4dfa67d8ff54110cae8b34826934f9b
parent3fcf19d592d5cb63eb209400b22055651e3c27d0
io_uring: mimimise io_cqring_wait_schedule

io_cqring_wait_schedule() is called after we started waiting on the cq
wq and set the state to TASK_INTERRUPTIBLE, for that reason we have to
constantly worry whether we has returned the state back to running or
not. Leave only quick checks in io_cqring_wait_schedule() and move the
rest including running task work to the callers. Note, we run tw in the
loop after the sched checks because of the fast path in the beginning of
the function.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/2814fabe75e2e019e7ca43ea07daa94564349805.1672916894.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring/io_uring.c