io_uring: add likely/unlikely in io_get_sqring()

The number of SQEs to submit is specified by a user, so io_get_sqring()
in most of the cases succeeds. Hint compilers about that.

Checking ASM genereted by gcc 9.2.0 for x64, there is one branch
misprediction.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 956a61cb0b132768a444174d13a9f0edfd24f85c..63e0448f3f8d956f416688663a8847eed0b1c903 100644 (file)
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -3129,11 +3129,11 @@ static bool io_get_sqring(struct io_ring_ctx *ctx, struct sqe_submit *s)
         */
        head = ctx->cached_sq_head;
        /* make sure SQ entry isn't read before tail */
-       if (head == smp_load_acquire(&rings->sq.tail))
+       if (unlikely(head == smp_load_acquire(&rings->sq.tail)))
                return false;
 
        head = READ_ONCE(sq_array[head & ctx->sq_mask]);
-       if (head < ctx->sq_entries) {
+       if (likely(head < ctx->sq_entries)) {
                s->ring_file = NULL;
                s->sqe = &ctx->sq_sqes[head];
                s->sequence = ctx->cached_sq_head;