io_uring: briefly loose locks while reaping events

It's not nice to hold @uring_lock for too long io_iopoll_reap_events().
For instance, the lock is needed to publish requests to @poll_list, and
that locks out tasks doing that for no good reason. Loose it
occasionally.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 332008f346e31a12cf65de3d0ee2e4a532474129..6e3169834bf7fb31e84a59f9db9f0473a3c82db4 100644 (file)
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -2069,8 +2069,13 @@ static void io_iopoll_reap_events(struct io_ring_ctx *ctx)
                /*
                 * Ensure we allow local-to-the-cpu processing to take place,
                 * in this case we need to ensure that we reap all events.
+                * Also let task_work, etc. to progress by releasing the mutex
                 */
-               cond_resched();
+               if (need_resched()) {
+                       mutex_unlock(&ctx->uring_lock);
+                       cond_resched();
+                       mutex_lock(&ctx->uring_lock);
+               }
        }
        mutex_unlock(&ctx->uring_lock);
 }