提交 c4b60d58 编写于 作者: P Pavel Begunkov 提交者: Xiaoguang Wang

io-wq: optimise locking in io_worker_handle_work()

to #28170604

commit 58e3931987377d3f4ec7bbc13e4ea0aab52dc6b0 upstream

There are 2 optimisations:
- Now, io_worker_handler_work() do io_assign_current_work() twice per
request, and each one adds lock/unlock(worker->lock) pair. The first is
to reset worker->cur_work to NULL, and the second to set a real work
shortly after. If there is a dependant work, set it immediately, that
effectively removes the extra NULL'ing.

- And there is no use in taking wqe->lock for linked works, as they are
not hashed now. Optimise it out.
Signed-off-by: NPavel Begunkov <asml.silence@gmail.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>
Acked-by: NJoseph Qi <joseph.qi@linux.alibaba.com>
Signed-off-by: NXiaoguang Wang <xiaoguang.wang@linux.alibaba.com>
上级 128bda4c
......@@ -477,7 +477,7 @@ static void io_worker_handle_work(struct io_worker *worker)
struct io_wq *wq = wqe->wq;
do {
struct io_wq_work *work, *old_work;
struct io_wq_work *work;
unsigned hash = -1U;
/*
......@@ -496,12 +496,13 @@ static void io_worker_handle_work(struct io_worker *worker)
spin_unlock_irq(&wqe->lock);
if (!work)
break;
io_assign_current_work(worker, work);
/* handle a whole dependent link */
do {
io_assign_current_work(worker, work);
io_impersonate_work(worker, work);
struct io_wq_work *old_work;
io_impersonate_work(worker, work);
/*
* OK to set IO_WQ_WORK_CANCEL even for uncancellable
* work, the worker function will do the right thing.
......@@ -514,10 +515,8 @@ static void io_worker_handle_work(struct io_worker *worker)
old_work = work;
work->func(&work);
spin_lock_irq(&worker->lock);
worker->cur_work = NULL;
spin_unlock_irq(&worker->lock);
work = (old_work == work) ? NULL : work;
io_assign_current_work(worker, work);
if (wq->put_work)
wq->put_work(old_work);
......@@ -530,7 +529,7 @@ static void io_worker_handle_work(struct io_worker *worker)
/* dependent work is not hashed */
hash = -1U;
}
} while (work && work != old_work);
} while (work);
spin_lock_irq(&wqe->lock);
} while (1);
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册