io_uring/rw: ensure reissue path is correctly handled for IOPOLL

The IOPOLL path posts CQEs when the io_kiocb is marked as completed,
so it cannot rely on the usual retry that non-IOPOLL requests do for
read/write requests.

If -EAGAIN is received and the request should be retried, go through
the normal completion path and let the normal flush logic catch it and
reissue it, like what is done for !IOPOLL reads or writes.

Fixes: d803d123948f ("io_uring/rw: handle -EAGAIN retry at IO completion time")
Reported-by: John Garry <john.g.garry@oracle.com>
Link: https://lore.kernel.org/io-uring/2b43ccfa-644d-4a09-8f8f-39ad71810f41@oracle.com/
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/io_uring/rw.c b/io_uring/rw.c
index 9edc6baebd01cf1246699aac7bbfc4b21c5df008..e5528cebcd06672a17466f7f15cfc15ec117e764 100644 (file)
--- a/io_uring/rw.c
+++ b/io_uring/rw.c
@@ -560,11 +560,10 @@ static void io_complete_rw_iopoll(struct kiocb *kiocb, long res)
        if (kiocb->ki_flags & IOCB_WRITE)
                io_req_end_write(req);
        if (unlikely(res != req->cqe.res)) {
-               if (res == -EAGAIN && io_rw_should_reissue(req)) {
+               if (res == -EAGAIN && io_rw_should_reissue(req))
                        req->flags |= REQ_F_REISSUE | REQ_F_BL_NO_RECYCLE;
-                       return;
-               }
-               req->cqe.res = res;
+               else
+                       req->cqe.res = res;
        }
 
        /* order with io_iopoll_complete() checking ->iopoll_completed */