• M
    mm/page_alloc: leave IRQs enabled for per-cpu page allocations · 57490774
    Mel Gorman 提交于
    The pcp_spin_lock_irqsave protecting the PCP lists is IRQ-safe as a task
    allocating from the PCP must not re-enter the allocator from IRQ context. 
    In each instance where IRQ-reentrancy is possible, the lock is acquired
    using pcp_spin_trylock_irqsave() even though IRQs are disabled and
    re-entrancy is impossible.
    
    Demote the lock to pcp_spin_lock avoids an IRQ disable/enable in the
    common case at the cost of some IRQ allocations taking a slower path.  If
    the PCP lists need to be refilled, the zone lock still needs to disable
    IRQs but that will only happen on PCP refill and drain.  If an IRQ is
    raised when a PCP allocation is in progress, the trylock will fail and
    fallback to using the buddy lists directly.  Note that this may not be a
    universal win if an interrupt-intensive workload also allocates heavily
    from interrupt context and contends heavily on the zone->lock as a result.
    
    [mgorman@techsingularity.net: migratetype might be wrong if a PCP was locked]
      Link: https://lkml.kernel.org/r/20221122131229.5263-2-mgorman@techsingularity.net
    [yuzhao@google.com: reported lockdep issue on IO completion from softirq]
    [hughd@google.com: fix list corruption, lock improvements, micro-optimsations]
    Link: https://lkml.kernel.org/r/20221118101714.19590-3-mgorman@techsingularity.netSigned-off-by: NMel Gorman <mgorman@techsingularity.net>
    Reviewed-by: NVlastimil Babka <vbabka@suse.cz>
    Cc: Marcelo Tosatti <mtosatti@redhat.com>
    Cc: Marek Szyprowski <m.szyprowski@samsung.com>
    Cc: Michal Hocko <mhocko@kernel.org>
    Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
    57490774
page_alloc.c 269.8 KB