1. 15 1月, 2009 4 次提交
    • C
      mutex: adaptive spinnning, performance tweaks · ac6e60ee
      Chris Mason 提交于
      Spin more agressively. This is less fair but also markedly faster.
      
      The numbers:
      
       * dbench 50 (higher is better):
        spin        1282MB/s
        v10         548MB/s
        v10 no wait 1868MB/s
      
       * 4k creates (numbers in files/second higher is better):
        spin        avg 200.60 median 193.20 std 19.71 high 305.93 low 186.82
        v10         avg 180.94 median 175.28 std 13.91 high 229.31 low 168.73
        v10 no wait avg 232.18 median 222.38 std 22.91 high 314.66 low 209.12
      
       * File stats (numbers in seconds, lower is better):
        spin        2.27s
        v10         5.1s
        v10 no wait 1.6s
      
      ( The source changes are smaller than they look, I just moved the
        need_resched checks in __mutex_lock_common after the cmpxchg. )
      Signed-off-by: NChris Mason <chris.mason@oracle.com>
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      ac6e60ee
    • P
      mutex: implement adaptive spinning · 0d66bf6d
      Peter Zijlstra 提交于
      Change mutex contention behaviour such that it will sometimes busy wait on
      acquisition - moving its behaviour closer to that of spinlocks.
      
      This concept got ported to mainline from the -rt tree, where it was originally
      implemented for rtmutexes by Steven Rostedt, based on work by Gregory Haskins.
      
      Testing with Ingo's test-mutex application (http://lkml.org/lkml/2006/1/8/50)
      gave a 345% boost for VFS scalability on my testbox:
      
       # ./test-mutex-shm V 16 10 | grep "^avg ops"
       avg ops/sec:               296604
      
       # ./test-mutex-shm V 16 10 | grep "^avg ops"
       avg ops/sec:               85870
      
      The key criteria for the busy wait is that the lock owner has to be running on
      a (different) cpu. The idea is that as long as the owner is running, there is a
      fair chance it'll release the lock soon, and thus we'll be better off spinning
      instead of blocking/scheduling.
      
      Since regular mutexes (as opposed to rtmutexes) do not atomically track the
      owner, we add the owner in a non-atomic fashion and deal with the races in
      the slowpath.
      
      Furthermore, to ease the testing of the performance impact of this new code,
      there is means to disable this behaviour runtime (without having to reboot
      the system), when scheduler debugging is enabled (CONFIG_SCHED_DEBUG=y),
      by issuing the following command:
      
       # echo NO_OWNER_SPIN > /debug/sched_features
      
      This command re-enables spinning again (this is also the default):
      
       # echo OWNER_SPIN > /debug/sched_features
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      0d66bf6d
    • P
      mutex: preemption fixes · 41719b03
      Peter Zijlstra 提交于
      The problem is that dropping the spinlock right before schedule is a voluntary
      preemption point and can cause a schedule, right after which we schedule again.
      
      Fix this inefficiency by keeping preemption disabled until we schedule, do this
      by explicity disabling preemption and providing a schedule() variant that
      assumes preemption is already disabled.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      41719b03
    • P
      mutex: small cleanup · 93d81d1a
      Peter Zijlstra 提交于
      Remove a local variable by combining an assingment and test in one.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      93d81d1a
  2. 14 1月, 2009 15 次提交
  3. 13 1月, 2009 21 次提交