rq-qos: use a mb for got_token

mainline inclusion
from mainline-5.3-rc2
commit ac38297f7038cd5b80d66f8809c7bbf5b70031f3
category: bugfix
bugzilla: 21211
CVE: NA

---------------------------

Oleg noticed that our checking of data.got_token is unsafe in the
cleanup case, and should really use a memory barrier.  Use a wmb on the
write side, and a rmb() on the read side.  We don't need one in the main
loop since we're saved by set_current_state().
Reviewed-by: NOleg Nesterov <oleg@redhat.com>
Signed-off-by: NJosef Bacik <josef@toxicpanda.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Conflicts:
	block/blk-rq-qos.c
[yan: the code was in __wbt_wait() and wbt_wake_function()]
Signed-off-by: NJason Yan <yanaijie@huawei.com>
Reviewed-by: NYufen Yu <yuyufen@huawei.com>
Signed-off-by: NYang Yingliang <yangyingliang@huawei.com>

block/blk-wbt.c

@@ -511,6 +511,7 @@ static int wbt_wake_function(struct wait_queue_entry *curr, unsigned int mode,
 		return -1;
 
 	data->got_token = true;
+	smp_wmb();
 	list_del_init(&curr->entry);
 	wake_up_process(data->task);
 	return 1;
@@ -545,6 +546,7 @@ static void __wbt_wait(struct rq_wb *rwb, enum wbt_flags wb_acct,
 	prepare_to_wait_exclusive(&rqw->wait, &data.wq, TASK_UNINTERRUPTIBLE);
 	has_sleeper = !wq_has_single_sleeper(&rqw->wait);
 	do {
+		/* The memory barrier in set_task_state saves us here. */
 		if (data.got_token)
 			break;
 
@@ -557,6 +559,7 @@ static void __wbt_wait(struct rq_wb *rwb, enum wbt_flags wb_acct,
 			 * which means we now have two. Put our local token
 			 * and wake anyone else potentially waiting for one.
 			 */
+			smp_rmb();
 			if (data.got_token)
 				wbt_rqw_done(rwb, rqw, wb_acct);
 			break;