• M
    mm: slab: optimize memcg_slab_free_hook() · b77d5b1b
    Muchun Song 提交于
    Most callers of memcg_slab_free_hook() already know the slab,  which could
    be passed to memcg_slab_free_hook() directly to reduce the overhead of an
    another call of virt_to_slab().  For bulk freeing of objects, the call of
    slab_objcgs() in the loop in memcg_slab_free_hook() is redundant as well.
    Rework memcg_slab_free_hook() and build_detached_freelist() to reduce
    those unnecessary overhead and make memcg_slab_free_hook() can handle bulk
    freeing in slab_free().
    
    Move the calling site of memcg_slab_free_hook() from do_slab_free() to
    slab_free() for slub to make the code clearer since the logic is weird
    (e.g. the caller need to judge whether it needs to call
    memcg_slab_free_hook()). It is easy to make mistakes like missing calling
    of memcg_slab_free_hook() like fixes of:
    
      commit d1b2cf6c ("mm: memcg/slab: uncharge during kmem_cache_free_bulk()")
      commit ae085d7f ("mm: kfence: fix missing objcg housekeeping for SLAB")
    
    This optimization is mainly for bulk objects freeing.  The following numbers
    is shown for 16-object freeing.
    
                               before      after
      kmem_cache_free_bulk:   ~430 ns     ~400 ns
    
    The overhead is reduced by about 7% for 16-object freeing.
    Signed-off-by: NMuchun Song <songmuchun@bytedance.com>
    Reviewed-by: NHyeonggon Yoo <42.hyeyoo@gmail.com>
    Link: https://lore.kernel.org/r/20220429123044.37885-1-songmuchun@bytedance.comSigned-off-by: NVlastimil Babka <vbabka@suse.cz>
    b77d5b1b
slab.h 22.8 KB