• C
    mm: slub: optimise the SLUB fast path to avoid pfmemalloc checks · 5091b74a
    Christoph Lameter 提交于
    This patch removes the check for pfmemalloc from the alloc hotpath and
    puts the logic after the election of a new per cpu slab.  For a pfmemalloc
    page we do not use the fast path but force the use of the slow path which
    is also used for the debug case.
    
    This has the side-effect of weakening pfmemalloc processing in the
    following way;
    
    1. A process that is allocating for network swap calls __slab_alloc.
       pfmemalloc_match is true so the freelist is loaded and c->freelist is
       now pointing to a pfmemalloc page.
    
    2. A process that is attempting normal allocations calls slab_alloc,
       finds the pfmemalloc page on the freelist and uses it because it did
       not check pfmemalloc_match()
    
    The patch allows non-pfmemalloc allocations to use pfmemalloc pages with
    the kmalloc slabs being the most vunerable caches on the grounds they
    are most likely to have a mix of pfmemalloc and !pfmemalloc requests. A
    later patch will still protect the system as processes will get throttled
    if the pfmemalloc reserves get depleted but performance will not degrade
    as smoothly.
    
    [mgorman@suse.de: Expanded changelog]
    Signed-off-by: NChristoph Lameter <cl@linux.com>
    Signed-off-by: NMel Gorman <mgorman@suse.de>
    Cc: David Miller <davem@davemloft.net>
    Cc: Neil Brown <neilb@suse.de>
    Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Cc: Mike Christie <michaelc@cs.wisc.edu>
    Cc: Eric B Munson <emunson@mgebm.net>
    Cc: Eric Dumazet <eric.dumazet@gmail.com>
    Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
    Cc: Mel Gorman <mgorman@suse.de>
    Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
    5091b74a
slub.c 129.7 KB