1. 08 9月, 2014 12 次提交
  2. 10 7月, 2014 1 次提交
  3. 24 6月, 2014 2 次提交
    • P
      rcu: Reduce overhead of cond_resched() checks for RCU · 4a81e832
      Paul E. McKenney 提交于
      Commit ac1bea85 (Make cond_resched() report RCU quiescent states)
      fixed a problem where a CPU looping in the kernel with but one runnable
      task would give RCU CPU stall warnings, even if the in-kernel loop
      contained cond_resched() calls.  Unfortunately, in so doing, it introduced
      performance regressions in Anton Blanchard's will-it-scale "open1" test.
      The problem appears to be not so much the increased cond_resched() path
      length as an increase in the rate at which grace periods complete, which
      increased per-update grace-period overhead.
      
      This commit takes a different approach to fixing this bug, mainly by
      moving the RCU-visible quiescent state from cond_resched() to
      rcu_note_context_switch(), and by further reducing the check to a
      simple non-zero test of a single per-CPU variable.  However, this
      approach requires that the force-quiescent-state processing send
      resched IPIs to the offending CPUs.  These will be sent only once
      the grace period has reached an age specified by the boot/sysfs
      parameter rcutree.jiffies_till_sched_qs, or once the grace period
      reaches an age halfway to the point at which RCU CPU stall warnings
      will be emitted, whichever comes first.
      Reported-by: NDave Hansen <dave.hansen@intel.com>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Andi Kleen <ak@linux.intel.com>
      Cc: Christoph Lameter <cl@gentwo.org>
      Cc: Mike Galbraith <umgwanakikbuti@gmail.com>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Reviewed-by: NJosh Triplett <josh@joshtriplett.org>
      [ paulmck: Made rcu_momentary_dyntick_idle() as suggested by the
        ktest build robot.  Also fixed smp_mb() comment as noted by
        Oleg Nesterov. ]
      
      Merge with e552592e (Reduce overhead of cond_resched() checks for RCU)
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      4a81e832
    • P
      rcu: Export debug_init_rcu_head() and and debug_init_rcu_head() · 546a9d85
      Paul E. McKenney 提交于
      Currently, call_rcu() relies on implicit allocation and initialization
      for the debug-objects handling of RCU callbacks.  If you hammer the
      kernel hard enough with Sasha's modified version of trinity, you can end
      up with the sl*b allocators recursing into themselves via this implicit
      call_rcu() allocation.
      
      This commit therefore exports the debug_init_rcu_head() and
      debug_rcu_head_free() functions, which permits the allocators to allocated
      and pre-initialize the debug-objects information, so that there no longer
      any need for call_rcu() to do that initialization, which in turn prevents
      the recursion into the memory allocators.
      Reported-by: NSasha Levin <sasha.levin@oracle.com>
      Suggested-by: NThomas Gleixner <tglx@linutronix.de>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Acked-by: NThomas Gleixner <tglx@linutronix.de>
      Looks-good-to: Christoph Lameter <cl@linux.com>
      546a9d85
  4. 20 5月, 2014 1 次提交
  5. 15 5月, 2014 1 次提交
    • P
      sched,rcu: Make cond_resched() report RCU quiescent states · ac1bea85
      Paul E. McKenney 提交于
      Given a CPU running a loop containing cond_resched(), with no
      other tasks runnable on that CPU, RCU will eventually report RCU
      CPU stall warnings due to lack of quiescent states.  Fortunately,
      every call to cond_resched() is a perfectly good quiescent state.
      Unfortunately, invoking rcu_note_context_switch() is a bit heavyweight
      for cond_resched(), especially given the need to disable preemption,
      and, for RCU-preempt, interrupts as well.
      
      This commit therefore maintains a per-CPU counter that causes
      cond_resched(), cond_resched_lock(), and cond_resched_softirq() to call
      rcu_note_context_switch(), but only about once per 256 invocations.
      This ratio was chosen in keeping with the relative time constants of
      RCU grace periods.
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Reviewed-by: NJosh Triplett <josh@joshtriplett.org>
      ac1bea85
  6. 26 2月, 2014 1 次提交
    • P
      rcu: Ensure kernel/rcu/rcu.h can be sourced/used stand-alone · 5cb5c6e1
      Paul Gortmaker 提交于
      The kbuild test bot uncovered an implicit dependence on the
      trace header being present before rcu.h in ia64 allmodconfig
      that looks like this:
      
      In file included from kernel/ksysfs.c:22:0:
      kernel/rcu/rcu.h: In function '__rcu_reclaim':
      kernel/rcu/rcu.h:107:3: error: implicit declaration of function 'trace_rcu_invoke_kfree_callback' [-Werror=implicit-function-declaration]
      kernel/rcu/rcu.h:112:3: error: implicit declaration of function 'trace_rcu_invoke_callback' [-Werror=implicit-function-declaration]
      cc1: some warnings being treated as errors
      
      Looking at other rcu.h users, we can find that they all
      were sourcing the trace header in advance of rcu.h itself,
      as seen in the context of this diff.  There were also some
      inconsistencies as to whether it was or wasn't sourced based
      on the parent tracing Kconfig.
      
      Rather than "fix" it at each use site, and have inconsistent
      use based on whether "#ifdef CONFIG_RCU_TRACE" was used or not,
      lets just source the trace header just once, in the actual consumer
      of it, which is rcu.h itself.  We include it unconditionally, as
      build testing shows us that is a hard requirement for some files.
      Reported-by: Nkbuild test robot <fengguang.wu@intel.com>
      Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      5cb5c6e1
  7. 18 2月, 2014 1 次提交
  8. 25 1月, 2014 1 次提交
    • O
      introduce __fcheck_files() to fix rcu_dereference_check_fdtable(), kill rcu_my_thread_group_empty() · a8d4b834
      Oleg Nesterov 提交于
      rcu_dereference_check_fdtable() looks very wrong,
      
      1. rcu_my_thread_group_empty() was added by 844b9a87 "vfs: fix
         RCU-lockdep false positive due to /proc" but it doesn't really
         fix the problem. A CLONE_THREAD (without CLONE_FILES) task can
         hit the same race with get_files_struct().
      
         And otoh rcu_my_thread_group_empty() can suppress the correct
         warning if the caller is the CLONE_FILES (without CLONE_THREAD)
         task.
      
      2. files->count == 1 check is not really right too. Even if this
         files_struct is not shared it is not safe to access it lockless
         unless the caller is the owner.
      
         Otoh, this check is sub-optimal. files->count == 0 always means
         it is safe to use it lockless even if files != current->files,
         but put_files_struct() has to take rcu_read_lock(). See the next
         patch.
      
      This patch removes the buggy checks and turns fcheck_files() into
      __fcheck_files() which uses rcu_dereference_raw(), the "unshared"
      callers, fget_light() and fget_raw_light(), can use it to avoid
      the warning from RCU-lockdep.
      
      fcheck_files() is trivially reimplemented as rcu_lockdep_assert()
      plus __fcheck_files().
      Signed-off-by: NOleg Nesterov <oleg@redhat.com>
      Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
      a8d4b834
  9. 10 12月, 2013 1 次提交
  10. 16 10月, 2013 1 次提交
  11. 25 9月, 2013 1 次提交
  12. 24 9月, 2013 1 次提交
  13. 05 9月, 2013 1 次提交
  14. 19 8月, 2013 1 次提交
    • P
      rcu: Simplify debug-objects fixups · 15100df8
      Paul E. McKenney 提交于
      The current debug-objects fixups are complex and heavyweight, and the
      fixups are not complete:  Even with the fixups, RCU's callback lists
      can still be corrupted.  This commit therefore strips the fixups down
      to their minimal form, eliminating two of the three.
      
      It would be even better if (for example) call_rcu() simply leaked
      any problematic callbacks, but for that to happen, the debug-objects
      system would need to inform its caller of suspicious situations.
      This is the subject of a later commit in this series.
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
      Cc: Sedat Dilek <sedat.dilek@gmail.com>
      Cc: Davidlohr Bueso <davidlohr.bueso@hp.com>
      Cc: Rik van Riel <riel@surriel.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Tested-by: NSedat Dilek <sedat.dilek@gmail.com>
      Reviewed-by: NJosh Triplett <josh@joshtriplett.org>
      15100df8
  15. 30 7月, 2013 1 次提交
    • S
      rcu: Add const annotation to char * for RCU tracepoints and functions · e66c33d5
      Steven Rostedt (Red Hat) 提交于
      All the RCU tracepoints and functions that reference char pointers do
      so with just 'char *' even though they do not modify the contents of
      the string itself. This will cause warnings if a const char * is used
      in one of these functions.
      
      The RCU tracepoints store the pointer to the string to refer back to them
      when the trace output is displayed. As this can be minutes, hours or
      even days later, those strings had better be constant.
      
      This change also opens the door to allow the RCU tracepoint strings and
      their addresses to be exported so that userspace tracing tools can
      translate the contents of the pointers of the RCU tracepoints.
      Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>
      e66c33d5
  16. 11 6月, 2013 2 次提交
  17. 29 1月, 2013 1 次提交
  18. 09 1月, 2013 1 次提交
    • P
      rcu: Reduce rcutorture tracing · 52494535
      Paul E. McKenney 提交于
      Currently, rcutorture traces every read-side access.  This can be
      problematic because even a two-minute rcutorture run on a two-CPU system
      can generate 28,853,363 reads.  Normally, only a failing read is of
      interest, so this commit traces adjusts rcutorture's tracing to only
      trace failing reads.  The resulting event tracing records the time
      and the ->completed value captured at the beginning of the RCU read-side
      critical section, allowing correlation with other event-tracing messages.
      Signed-off-by: NPaul E. McKenney <paul.mckenney@linaro.org>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Reviewed-by: NJosh Triplett <josh@joshtriplett.org>
      [ paulmck: Add fix to build problem located by Randy Dunlap based on
        diagnosis by Steven Rostedt. ]
      52494535
  19. 24 10月, 2012 1 次提交
  20. 23 9月, 2012 1 次提交
  21. 03 7月, 2012 1 次提交
  22. 03 5月, 2012 1 次提交
    • P
      rcu: Make exit_rcu() more precise and consolidate · 9dd8fb16
      Paul E. McKenney 提交于
      When running preemptible RCU, if a task exits in an RCU read-side
      critical section having blocked within that same RCU read-side critical
      section, the task must be removed from the list of tasks blocking a
      grace period (perhaps the current grace period, perhaps the next grace
      period, depending on timing).  The exit() path invokes exit_rcu() to
      do this cleanup.
      
      However, the current implementation of exit_rcu() needlessly does the
      cleanup even if the task did not block within the current RCU read-side
      critical section, which wastes time and needlessly increases the size
      of the state space.  Fix this by only doing the cleanup if the current
      task is actually on the list of tasks blocking some grace period.
      
      While we are at it, consolidate the two identical exit_rcu() functions
      into a single function.
      Signed-off-by: NPaul E. McKenney <paul.mckenney@linaro.org>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Tested-by: NLinus Torvalds <torvalds@linux-foundation.org>
      
      Conflicts:
      
      	kernel/rcupdate.c
      9dd8fb16
  23. 22 2月, 2012 1 次提交
    • P
      rcu: Check for illegal use of RCU from offlined CPUs · c0d6d01b
      Paul E. McKenney 提交于
      Although it is legal to use RCU during early boot, it is anything
      but legal to use RCU at runtime from an offlined CPU.  After all, RCU
      explicitly ignores offlined CPUs.  This commit therefore adds checks
      for runtime use of RCU from offlined CPUs.
      
      These checks are not perfect, in particular, they can be subverted
      through use of things like rcu_dereference_raw().  Note that it is not
      possible to put checks in rcu_read_lock() and friends due to the fact
      that these primitives are used in code that might be used under either
      RCU or lock-based protection, which means that checking rcu_read_lock()
      gets you fat piles of false positives.
      Signed-off-by: NPaul E. McKenney <paul.mckenney@linaro.org>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      c0d6d01b
  24. 12 12月, 2011 2 次提交
  25. 31 10月, 2011 1 次提交
  26. 29 9月, 2011 1 次提交