1. 25 6月, 2020 5 次提交
    • J
      io_uring: pass in completion state to appropriate issue side handlers · 229a7b63
      Jens Axboe 提交于
      Provide the completion state to the handlers that we know can complete
      inline, so they can utilize this for batching completions.
      
      Cap the max batch count at 32. This should be enough to provide a good
      amortization of the cost of the lock+commit dance for completions, while
      still being low enough not to cause any real latency issues for SQPOLL
      applications.
      
      Xuan Zhuo <xuanzhuo@linux.alibaba.com> reports that this changes his
      profile from:
      
      17.97% [kernel] [k] copy_user_generic_unrolled
      13.92% [kernel] [k] io_commit_cqring
      11.04% [kernel] [k] __io_cqring_fill_event
      10.33% [kernel] [k] udp_recvmsg
       5.94% [kernel] [k] skb_release_data
       4.31% [kernel] [k] udp_rmem_release
       2.68% [kernel] [k] __check_object_size
       2.24% [kernel] [k] __slab_free
       2.22% [kernel] [k] _raw_spin_lock_bh
       2.21% [kernel] [k] kmem_cache_free
       2.13% [kernel] [k] free_pcppages_bulk
       1.83% [kernel] [k] io_submit_sqes
       1.38% [kernel] [k] page_frag_free
       1.31% [kernel] [k] inet_recvmsg
      
      to
      
      19.99% [kernel] [k] copy_user_generic_unrolled
      11.63% [kernel] [k] skb_release_data
       9.36% [kernel] [k] udp_rmem_release
       8.64% [kernel] [k] udp_recvmsg
       6.21% [kernel] [k] __slab_free
       4.39% [kernel] [k] __check_object_size
       3.64% [kernel] [k] free_pcppages_bulk
       2.41% [kernel] [k] kmem_cache_free
       2.00% [kernel] [k] io_submit_sqes
       1.95% [kernel] [k] page_frag_free
       1.54% [kernel] [k] io_put_req
      [...]
       0.07% [kernel] [k] io_commit_cqring
       0.44% [kernel] [k] __io_cqring_fill_event
      Signed-off-by: NJens Axboe <axboe@kernel.dk>
      229a7b63
    • J
      io_uring: pass down completion state on the issue side · f13fad7b
      Jens Axboe 提交于
      No functional changes in this patch, just in preparation for having the
      completion state be available on the issue side. Later on, this will
      allow requests that complete inline to be completed in batches.
      Signed-off-by: NJens Axboe <axboe@kernel.dk>
      f13fad7b
    • J
      io_uring: add 'io_comp_state' to struct io_submit_state · 013538bd
      Jens Axboe 提交于
      No functional changes in this patch, just in preparation for passing back
      pending completions to the caller and completing them in a batched
      fashion.
      Signed-off-by: NJens Axboe <axboe@kernel.dk>
      013538bd
    • J
      io_uring: provide generic io_req_complete() helper · e1e16097
      Jens Axboe 提交于
      We have lots of callers of:
      
      io_cqring_add_event(req, result);
      io_put_req(req);
      
      Provide a helper that does this for us. It helps clean up the code, and
      also provides a more convenient location for us to change the completion
      handling.
      Signed-off-by: NJens Axboe <axboe@kernel.dk>
      e1e16097
    • P
      io_uring: fix NULL-mm for linked reqs · d3cac64c
      Pavel Begunkov 提交于
      __io_queue_sqe() tries to handle all request of a link,
      so it's not enough to grab mm in io_sq_thread_acquire_mm()
      based just on the head.
      
      Don't check req->needs_mm and do it always.
      Signed-off-by: NPavel Begunkov <asml.silence@gmail.com>
      d3cac64c
  2. 22 6月, 2020 32 次提交
  3. 21 6月, 2020 3 次提交