1. 22 2月, 2012 7 次提交
  2. 12 12月, 2011 19 次提交
  3. 31 10月, 2011 1 次提交
  4. 29 9月, 2011 13 次提交
    • P
      rcu: Move propagation of ->completed from rcu_start_gp() to rcu_report_qs_rsp() · afe24b12
      Paul E. McKenney 提交于
      It is possible for the CPU that noted the end of the prior grace period
      to not need a new one, and therefore to decide to propagate ->completed
      throughout the rcu_node tree without starting another grace period.
      However, in so doing, it releases the root rcu_node structure's lock,
      which can allow some other CPU to start another grace period.  The first
      CPU will be propagating ->completed in parallel with the second CPU
      initializing the rcu_node tree for the new grace period.  In theory
      this is harmless, but in practice we need to keep things simple.
      
      This commit therefore moves the propagation of ->completed to
      rcu_report_qs_rsp(), and refrains from marking the old grace period
      as having been completed until it has finished doing this.  This
      prevents anyone from starting a new grace period concurrently with
      marking the old grace period as having been completed.
      
      Of course, the optimization where a CPU needing a new grace period
      doesn't bother marking the old one completed is still in effect:
      In that case, the marking happens implicitly as part of initializing
      the new grace period.
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      afe24b12
    • P
      rcu: Remove rcu_needs_cpu_flush() to avoid false quiescent states · e90c53d3
      Paul E. McKenney 提交于
      The purpose of rcu_needs_cpu_flush() was to iterate on pushing the
      current grace period in order to help the current CPU enter dyntick-idle
      mode.  However, this can result in failures if the CPU starts entering
      dyntick-idle mode, but then backs out.  In this case, the call to
      rcu_pending() from rcu_needs_cpu_flush() might end up announcing a
      non-existing quiescent state.
      
      This commit therefore removes rcu_needs_cpu_flush() in favor of letting
      the dyntick-idle machinery at the end of the softirq handler push the
      loop along via its call to rcu_pending().
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      e90c53d3
    • M
      rcu: Wire up RCU_BOOST_PRIO for rcutree · 5b61b0ba
      Mike Galbraith 提交于
      RCU boost threads start life at RCU_BOOST_PRIO, while others remain
      at RCU_KTHREAD_PRIO.  While here, change thread names to match other
      kthreads, and adjust rcu_yield() to not override the priority set by
      the user.  This last change sets the stage for runtime changes to
      priority in the -rt tree.
      Signed-off-by: NMike Galbraith <efault@gmx.de>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      5b61b0ba
    • P
      rcu: Avoid having just-onlined CPU resched itself when RCU is idle · 06ae115a
      Paul E. McKenney 提交于
      CPUs set rdp->qs_pending when coming online to resolve races with
      grace-period start.  However, this means that if RCU is idle, the
      just-onlined CPU might needlessly send itself resched IPIs.  Adjust
      the online-CPU initialization to avoid this, and also to correctly
      cause the CPU to respond to the current grace period if needed.
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Tested-by: NJosh Boyer <jwboyer@redhat.com>
      Tested-by: NChristian Hoffmann <email@christianhoffmann.info>
      06ae115a
    • P
      rcu: Suppress NMI backtraces when stall ends before dump · 9bc8b558
      Paul E. McKenney 提交于
      It is possible for an RCU CPU stall to end just as it is detected, in
      which case the current code will uselessly dump all CPU's stacks.
      This commit therefore checks for this condition and refrains from
      sending needless NMIs.
      
      And yes, the stall might also end just after we checked all CPUs and
      tasks, but in that case we would at least have given some clue as
      to which CPU/task was at fault.
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      9bc8b558
    • P
      rcu: Prohibit grace periods during early boot · 037067a1
      Paul E. McKenney 提交于
      Greater use of RCU during early boot (before the scheduler is operating)
      is causing RCU to attempt to start grace periods during that time, which
      in turn is resulting in both RCU and the callback functions attempting
      to use the scheduler before it is ready.
      
      This commit prevents these problems by prohibiting RCU grace periods
      until after the scheduler has spawned the first non-idle task.
      Signed-off-by: NPaul E. McKenney <paul.mckenney@linaro.org>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      037067a1
    • P
      rcu: Prevent early boot set_need_resched() from __rcu_pending() · 5c51dd73
      Paul E. McKenney 提交于
      There isn't a whole lot of point in poking the scheduler before there
      are other tasks to switch to.  This commit therefore adds a check
      for rcu_scheduler_fully_active in __rcu_pending() to suppress any
      pre-scheduler calls to set_need_resched().  The downside of this approach
      is additional runtime overhead in a reasonably hot code path.
      Signed-off-by: NPaul E. McKenney <paul.mckenney@linaro.org>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      5c51dd73
    • P
      rcu: Dump local stack if cannot dump all CPUs' stacks · 4627e240
      Paul E. McKenney 提交于
      The trigger_all_cpu_backtrace() function is a no-op in architectures that
      do not define arch_trigger_all_cpu_backtrace.  On such architectures, RCU
      CPU stall warning messages contain no stack trace information, which makes
      debugging quite difficult.  This commit therefore substitutes dump_stack()
      for architectures that do not define arch_trigger_all_cpu_backtrace,
      so that at least the local CPU's stack is dumped as part of the RCU CPU
      stall warning message.
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      4627e240
    • P
      rcu: Make rcu_implicit_dynticks_qs() locals be correct size · 7eb4f455
      Paul E. McKenney 提交于
      When the ->dynticks field in the rcu_dynticks structure changed to an
      atomic_t, its size on 64-bit systems changed from 64 bits to 32 bits.
      The local variables in rcu_implicit_dynticks_qs() need to change as
      well, hence this commit.
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      7eb4f455
    • P
      rcu: Eliminate in_irq() checks in rcu_enter_nohz() · 717f98f0
      Paul E. McKenney 提交于
      The in_irq() check in rcu_enter_nohz() is redundant because if we really
      are in an interrupt, the attempt to re-enter dyntick-idle mode will invoke
      rcu_needs_cpu() in any case, which will force the check for RCU callbacks.
      So this commit removes the check along with the set_need_resched().
      Suggested-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      717f98f0
    • P
      rcu: Simplify quiescent-state accounting · e4cc1f22
      Paul E. McKenney 提交于
      There is often a delay between the time that a CPU passes through a
      quiescent state and the time that this quiescent state is reported to the
      RCU core.  It is quite possible that the grace period ended before the
      quiescent state could be reported, for example, some other CPU might have
      deduced that this CPU passed through dyntick-idle mode.  It is critically
      important that quiescent state be counted only against the grace period
      that was in effect at the time that the quiescent state was detected.
      
      Previously, this was handled by recording the number of the last grace
      period to complete when passing through a quiescent state.  The RCU
      core then checks this number against the current value, and rejects
      the quiescent state if there is a mismatch.  However, one additional
      possibility must be accounted for, namely that the quiescent state was
      recorded after the prior grace period completed but before the current
      grace period started.  In this case, the RCU core must reject the
      quiescent state, but the recorded number will match.  This is handled
      when the CPU becomes aware of a new grace period -- at that point,
      it invalidates any prior quiescent state.
      
      This works, but is a bit indirect.  The new approach records the current
      grace period, and the RCU core checks to see (1) that this is still the
      current grace period and (2) that this grace period has not yet ended.
      This approach simplifies reasoning about correctness, and this commit
      changes over to this new approach.
      Signed-off-by: NPaul E. McKenney <paul.mckenney@linaro.org>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      e4cc1f22
    • P
      rcu: Add grace-period, quiescent-state, and call_rcu trace events · d4c08f2a
      Paul E. McKenney 提交于
      Add trace events to record grace-period start and end, quiescent states,
      CPUs noticing grace-period start and end, grace-period initialization,
      call_rcu() invocation, tasks blocking in RCU read-side critical sections,
      tasks exiting those same critical sections, force_quiescent_state()
      detection of dyntick-idle and offline CPUs, CPUs entering and leaving
      dyntick-idle mode (except from NMIs), CPUs coming online and going
      offline, and CPUs being kicked for staying in dyntick-idle mode for too
      long (as in many weeks, even on 32-bit systems).
      Signed-off-by: NPaul E. McKenney <paul.mckenney@linaro.org>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      
      rcu: Add the rcu flavor to callback trace events
      
      The earlier trace events for registering RCU callbacks and for invoking
      them did not include the RCU flavor (rcu_bh, rcu_preempt, or rcu_sched).
      This commit adds the RCU flavor to those trace events.
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      d4c08f2a
    • P
      rcu: Update comments to reflect softirqs vs. kthreads · e0f23060
      Paul E. McKenney 提交于
      We now have kthreads only for flavors of RCU that support boosting,
      so update the now-misleading comments accordingly.
      Signed-off-by: NPaul E. McKenney <paul.mckenney@linaro.org>
      Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      e0f23060