From 21a4ae0ca5aeb13575eedde95a36640198bd2699 Mon Sep 17 00:00:00 2001 From: Pavel Begunkov Date: Thu, 15 Apr 2021 17:33:52 +0800 Subject: [PATCH] io_uring: add likely/unlikely in io_get_sqring() mainline inclusion from mainline-5.5-rc1 commit 9835d6fafba58e6d9386a6d5af800789bdb52e5b category: feature bugzilla: https://bugzilla.openeuler.org/show_bug.cgi?id=27 CVE: NA --------------------------- The number of SQEs to submit is specified by a user, so io_get_sqring() in most of the cases succeeds. Hint compilers about that. Checking ASM genereted by gcc 9.2.0 for x64, there is one branch misprediction. Signed-off-by: Pavel Begunkov Signed-off-by: Jens Axboe Signed-off-by: Zhihao Cheng Signed-off-by: yangerkun Reviewed-by: zhangyi (F) Signed-off-by: Cheng Jian --- fs/io_uring.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index bfacf7a8954b..d7ea7e0ee473 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -3128,11 +3128,11 @@ static bool io_get_sqring(struct io_ring_ctx *ctx, struct sqe_submit *s) */ head = ctx->cached_sq_head; /* make sure SQ entry isn't read before tail */ - if (head == smp_load_acquire(&rings->sq.tail)) + if (unlikely(head == smp_load_acquire(&rings->sq.tail))) return false; head = READ_ONCE(sq_array[head & ctx->sq_mask]); - if (head < ctx->sq_entries) { + if (likely(head < ctx->sq_entries)) { s->ring_file = NULL; s->sqe = &ctx->sq_sqes[head]; s->sequence = ctx->cached_sq_head; -- GitLab