io_uring: ensure overflow entries are dropped when ring is exiting

A previous consolidation cleanup missed handling the case where the ring
is dying, and __io_cqring_overflow_flush() doesn't flush entries if the
CQ ring is already full. This is fine for the normal CQE overflow
flushing, but if the ring is going away, we need to flush everything,
even if it means simply freeing the overflown entries.

Fixes: 6c948ec44b29 ("io_uring: consolidate overflow flushing")
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index c4419eef7e63d4895af8ea3e1ba13f052af6ea27..3c9087f37c43527ce39cadabd671c80778b3b2e5 100644 (file)
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -674,7 +674,8 @@ static void __io_cqring_overflow_flush(struct io_ring_ctx *ctx, bool dying)
 
        lockdep_assert_held(&ctx->uring_lock);
 
-       if (__io_cqring_events(ctx) == ctx->cq_entries)
+       /* don't abort if we're dying, entries must get freed */
+       if (!dying && __io_cqring_events(ctx) == ctx->cq_entries)
                return;
 
        if (ctx->flags & IORING_SETUP_CQE32)