• J
    io_uring: pass in completion state to appropriate issue side handlers · 229a7b63
    Jens Axboe 提交于
    Provide the completion state to the handlers that we know can complete
    inline, so they can utilize this for batching completions.
    
    Cap the max batch count at 32. This should be enough to provide a good
    amortization of the cost of the lock+commit dance for completions, while
    still being low enough not to cause any real latency issues for SQPOLL
    applications.
    
    Xuan Zhuo <xuanzhuo@linux.alibaba.com> reports that this changes his
    profile from:
    
    17.97% [kernel] [k] copy_user_generic_unrolled
    13.92% [kernel] [k] io_commit_cqring
    11.04% [kernel] [k] __io_cqring_fill_event
    10.33% [kernel] [k] udp_recvmsg
     5.94% [kernel] [k] skb_release_data
     4.31% [kernel] [k] udp_rmem_release
     2.68% [kernel] [k] __check_object_size
     2.24% [kernel] [k] __slab_free
     2.22% [kernel] [k] _raw_spin_lock_bh
     2.21% [kernel] [k] kmem_cache_free
     2.13% [kernel] [k] free_pcppages_bulk
     1.83% [kernel] [k] io_submit_sqes
     1.38% [kernel] [k] page_frag_free
     1.31% [kernel] [k] inet_recvmsg
    
    to
    
    19.99% [kernel] [k] copy_user_generic_unrolled
    11.63% [kernel] [k] skb_release_data
     9.36% [kernel] [k] udp_rmem_release
     8.64% [kernel] [k] udp_recvmsg
     6.21% [kernel] [k] __slab_free
     4.39% [kernel] [k] __check_object_size
     3.64% [kernel] [k] free_pcppages_bulk
     2.41% [kernel] [k] kmem_cache_free
     2.00% [kernel] [k] io_submit_sqes
     1.95% [kernel] [k] page_frag_free
     1.54% [kernel] [k] io_put_req
    [...]
     0.07% [kernel] [k] io_commit_cqring
     0.44% [kernel] [k] __io_cqring_fill_event
    Signed-off-by: NJens Axboe <axboe@kernel.dk>
    229a7b63
io_uring.c 201.2 KB