1. 11 7月, 2016 1 次提交
  2. 02 3月, 2016 1 次提交
    • F
      posix-cpu-timers: Migrate to use new tick dependency mask model · b7878300
      Frederic Weisbecker 提交于
      Instead of providing asynchronous checks for the nohz subsystem to verify
      posix cpu timers tick dependency, migrate the latter to the new mask.
      
      In order to keep track of the running timers and expose the tick
      dependency accordingly, we must probe the timers queuing and dequeuing
      on threads and process lists.
      
      Unfortunately it implies both task and signal level dependencies. We
      should be able to further optimize this and merge all that on the task
      level dependency, at the cost of a bit of complexity and may be overhead.
      Reviewed-by: NChris Metcalf <cmetcalf@ezchip.com>
      Cc: Christoph Lameter <cl@linux.com>
      Cc: Chris Metcalf <cmetcalf@ezchip.com>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Luiz Capitulino <lcapitulino@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Rik van Riel <riel@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Viresh Kumar <viresh.kumar@linaro.org>
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      b7878300
  3. 15 10月, 2015 4 次提交
  4. 08 5月, 2015 3 次提交
    • J
      sched, timer: Use the atomic task_cputime in thread_group_cputimer · 71107445
      Jason Low 提交于
      Recent optimizations were made to thread_group_cputimer to improve its
      scalability by keeping track of cputime stats without a lock. However,
      the values were open coded to the structure, causing them to be at
      a different abstraction level from the regular task_cputime structure.
      Furthermore, any subsequent similar optimizations would not be able to
      share the new code, since they are specific to thread_group_cputimer.
      
      This patch adds the new task_cputime_atomic data structure (introduced in
      the previous patch in the series) to thread_group_cputimer for keeping
      track of the cputime atomically, which also helps generalize the code.
      Suggested-by: NIngo Molnar <mingo@kernel.org>
      Signed-off-by: NJason Low <jason.low2@hp.com>
      Signed-off-by: NPeter Zijlstra (Intel) <peterz@infradead.org>
      Acked-by: NThomas Gleixner <tglx@linutronix.de>
      Acked-by: NRik van Riel <riel@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Aswin Chandramouleeswaran <aswin@hp.com>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Davidlohr Bueso <dave@stgolabs.net>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Mike Galbraith <umgwanakikbuti@gmail.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Preeti U Murthy <preeti@linux.vnet.ibm.com>
      Cc: Scott J Norton <scott.norton@hp.com>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Waiman Long <Waiman.Long@hp.com>
      Link: http://lkml.kernel.org/r/1430251224-5764-6-git-send-email-jason.low2@hp.comSigned-off-by: NIngo Molnar <mingo@kernel.org>
      71107445
    • J
      sched, timer: Replace spinlocks with atomics in thread_group_cputimer(), to improve scalability · 1018016c
      Jason Low 提交于
      While running a database workload, we found a scalability issue with itimers.
      
      Much of the problem was caused by the thread_group_cputimer spinlock.
      Each time we account for group system/user time, we need to obtain a
      thread_group_cputimer's spinlock to update the timers. On larger systems
      (such as a 16 socket machine), this caused more than 30% of total time
      spent trying to obtain this kernel lock to update these group timer stats.
      
      This patch converts the timers to 64-bit atomic variables and use
      atomic add to update them without a lock. With this patch, the percent
      of total time spent updating thread group cputimer timers was reduced
      from 30% down to less than 1%.
      
      Note: On 32-bit systems using the generic 64-bit atomics, this causes
      sample_group_cputimer() to take locks 3 times instead of just 1 time.
      However, we tested this patch on a 32-bit system ARM system using the
      generic atomics and did not find the overhead to be much of an issue.
      An explanation for why this isn't an issue is that 32-bit systems usually
      have small numbers of CPUs, and cacheline contention from extra spinlocks
      called periodically is not really apparent on smaller systems.
      Signed-off-by: NJason Low <jason.low2@hp.com>
      Signed-off-by: NPeter Zijlstra (Intel) <peterz@infradead.org>
      Acked-by: NThomas Gleixner <tglx@linutronix.de>
      Acked-by: NRik van Riel <riel@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Aswin Chandramouleeswaran <aswin@hp.com>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Davidlohr Bueso <dave@stgolabs.net>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Mike Galbraith <umgwanakikbuti@gmail.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Preeti U Murthy <preeti@linux.vnet.ibm.com>
      Cc: Scott J Norton <scott.norton@hp.com>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Waiman Long <Waiman.Long@hp.com>
      Link: http://lkml.kernel.org/r/1430251224-5764-4-git-send-email-jason.low2@hp.comSigned-off-by: NIngo Molnar <mingo@kernel.org>
      1018016c
    • J
      sched, timer: Convert usages of ACCESS_ONCE() in the scheduler to READ_ONCE()/WRITE_ONCE() · 316c1608
      Jason Low 提交于
      ACCESS_ONCE doesn't work reliably on non-scalar types. This patch removes
      the rest of the existing usages of ACCESS_ONCE() in the scheduler, and use
      the new READ_ONCE() and WRITE_ONCE() APIs as appropriate.
      Signed-off-by: NJason Low <jason.low2@hp.com>
      Signed-off-by: NPeter Zijlstra (Intel) <peterz@infradead.org>
      Acked-by: NThomas Gleixner <tglx@linutronix.de>
      Acked-by: NRik van Riel <riel@redhat.com>
      Acked-by: NWaiman Long <Waiman.Long@hp.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Aswin Chandramouleeswaran <aswin@hp.com>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Davidlohr Bueso <dave@stgolabs.net>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: H. Peter Anvin <hpa@zytor.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Mel Gorman <mgorman@suse.de>
      Cc: Mike Galbraith <umgwanakikbuti@gmail.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Preeti U Murthy <preeti@linux.vnet.ibm.com>
      Cc: Scott J Norton <scott.norton@hp.com>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Link: http://lkml.kernel.org/r/1430251224-5764-2-git-send-email-jason.low2@hp.comSigned-off-by: NIngo Molnar <mingo@kernel.org>
      316c1608
  5. 13 2月, 2015 1 次提交
    • A
      all arches, signal: move restart_block to struct task_struct · f56141e3
      Andy Lutomirski 提交于
      If an attacker can cause a controlled kernel stack overflow, overwriting
      the restart block is a very juicy exploit target.  This is because the
      restart_block is held in the same memory allocation as the kernel stack.
      
      Moving the restart block to struct task_struct prevents this exploit by
      making the restart_block harder to locate.
      
      Note that there are other fields in thread_info that are also easy
      targets, at least on some architectures.
      
      It's also a decent simplification, since the restart code is more or less
      identical on all architectures.
      
      [james.hogan@imgtec.com: metag: align thread_info::supervisor_stack]
      Signed-off-by: NAndy Lutomirski <luto@amacapital.net>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Al Viro <viro@zeniv.linux.org.uk>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Kees Cook <keescook@chromium.org>
      Cc: David Miller <davem@davemloft.net>
      Acked-by: NRichard Weinberger <richard@nod.at>
      Cc: Richard Henderson <rth@twiddle.net>
      Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
      Cc: Matt Turner <mattst88@gmail.com>
      Cc: Vineet Gupta <vgupta@synopsys.com>
      Cc: Russell King <rmk@arm.linux.org.uk>
      Cc: Catalin Marinas <catalin.marinas@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: Haavard Skinnemoen <hskinnemoen@gmail.com>
      Cc: Hans-Christian Egtvedt <egtvedt@samfundet.no>
      Cc: Steven Miao <realmz6@gmail.com>
      Cc: Mark Salter <msalter@redhat.com>
      Cc: Aurelien Jacquiot <a-jacquiot@ti.com>
      Cc: Mikael Starvik <starvik@axis.com>
      Cc: Jesper Nilsson <jesper.nilsson@axis.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Richard Kuo <rkuo@codeaurora.org>
      Cc: "Luck, Tony" <tony.luck@intel.com>
      Cc: Geert Uytterhoeven <geert@linux-m68k.org>
      Cc: Michal Simek <monstr@monstr.eu>
      Cc: Ralf Baechle <ralf@linux-mips.org>
      Cc: Jonas Bonn <jonas@southpole.se>
      Cc: "James E.J. Bottomley" <jejb@parisc-linux.org>
      Cc: Helge Deller <deller@gmx.de>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Paul Mackerras <paulus@samba.org>
      Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc)
      Tested-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc)
      Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
      Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
      Cc: Chen Liqin <liqin.linux@gmail.com>
      Cc: Lennox Wu <lennox.wu@gmail.com>
      Cc: Chris Metcalf <cmetcalf@ezchip.com>
      Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
      Cc: Chris Zankel <chris@zankel.net>
      Cc: Max Filippov <jcmvbkbc@gmail.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Guenter Roeck <linux@roeck-us.net>
      Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
      f56141e3
  6. 16 11月, 2014 1 次提交
  7. 08 9月, 2014 1 次提交
    • R
      time, signal: Protect resource use statistics with seqlock · e78c3496
      Rik van Riel 提交于
      Both times() and clock_gettime(CLOCK_PROCESS_CPUTIME_ID) have scalability
      issues on large systems, due to both functions being serialized with a
      lock.
      
      The lock protects against reporting a wrong value, due to a thread in the
      task group exiting, its statistics reporting up to the signal struct, and
      that exited task's statistics being counted twice (or not at all).
      
      Protecting that with a lock results in times() and clock_gettime() being
      completely serialized on large systems.
      
      This can be fixed by using a seqlock around the events that gather and
      propagate statistics. As an additional benefit, the protection code can
      be moved into thread_group_cputime(), slightly simplifying the calling
      functions.
      
      In the case of posix_cpu_clock_get_task() things can be simplified a
      lot, because the calling function already ensures that the task sticks
      around, and the rest is now taken care of in thread_group_cputime().
      
      This way the statistics reporting code can run lockless.
      Signed-off-by: NRik van Riel <riel@redhat.com>
      Signed-off-by: NPeter Zijlstra (Intel) <peterz@infradead.org>
      Cc: Alex Thorlton <athorlton@sgi.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Daeseok Youn <daeseok.youn@gmail.com>
      Cc: David Rientjes <rientjes@google.com>
      Cc: Dongsheng Yang <yangds.fnst@cn.fujitsu.com>
      Cc: Geert Uytterhoeven <geert@linux-m68k.org>
      Cc: Guillaume Morin <guillaume@morinfr.org>
      Cc: Ionut Alexa <ionut.m.alexa@gmail.com>
      Cc: Kees Cook <keescook@chromium.org>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Li Zefan <lizefan@huawei.com>
      Cc: Michal Hocko <mhocko@suse.cz>
      Cc: Michal Schmidt <mschmidt@redhat.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Vladimir Davydov <vdavydov@parallels.com>
      Cc: umgwanakikbuti@gmail.com
      Cc: fweisbec@gmail.com
      Cc: srao@redhat.com
      Cc: lwoodman@redhat.com
      Cc: atheurer@redhat.com
      Link: http://lkml.kernel.org/r/20140816134010.26a9b572@annuminas.surriel.comSigned-off-by: NIngo Molnar <mingo@kernel.org>
      e78c3496
  8. 23 6月, 2014 1 次提交
  9. 09 12月, 2013 10 次提交
    • F
      posix-timers: Convert abuses of BUG_ON to WARN_ON · 531f64fd
      Frederic Weisbecker 提交于
      The posix cpu timers code makes a heavy use of BUG_ON()
      but none of these concern fatal issues that require
      to stop the machine. So let's just warn the user when
      some internal state slips out of our hands.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Kosaki Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      531f64fd
    • F
      posix-timers: Remove remaining uses of tasklist_lock · e73d84e3
      Frederic Weisbecker 提交于
      The remaining uses of tasklist_lock were mostly about synchronizing
      against sighand modifications, getting coherent and safe group samples
      and also thread/process wide timers list handling.
      
      All of this is already safely synchronizable with the target's
      sighand lock. Let's use it on these places instead.
      
      Also update the comments about locking.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Kosaki Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      e73d84e3
    • F
      posix-timers: Use sighand lock instead of tasklist_lock on timer deletion · 3d7a1427
      Frederic Weisbecker 提交于
      Timer deletion doesn't need the tasklist lock.
      We need to protect against:
      
      * concurrent access to the lists p->cputime_expires and
        p->sighand->cputime_expires
      
      * task reaping that may also delete the timer list entry
      
      * timer firing
      
      We already hold the timer lock which protects us against concurrent
      timer firing.
      
      The rest only need the targets sighand to be locked.
      So hold it and drop the use of tasklist_lock there.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Kosaki Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      3d7a1427
    • F
      posix-timers: Use sighand lock instead of tasklist_lock for task clock sample · 50875788
      Frederic Weisbecker 提交于
      There is no need for the tasklist_lock just to take a process
      wide clock sample.
      
      All we need is to get a coherent sample that doesn't race with
      exit() and exec():
      
      * exit() may be concurrently reaping a task and flushing its time
      
      * sighand is unstable under exit() and exec(), and the latter also
        result in group leader that can change
      
      To protect against these, locking the target's sighand is enough.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Kosaki Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      50875788
    • F
      posix-timers: Consolidate posix_cpu_clock_get() · 33ab0fec
      Frederic Weisbecker 提交于
      Consolidate the clock sampling common code used for both local
      and remote targets.
      
      Note that this introduces a tiny user ABI change: if a
      PID is passed to clock_gettime() along the clockid,
      we used to forbid a process wide clock sample when that
      PID doesn't belong to a group leader. Now after this patch
      we allow process wide clock samples if that PID belongs to
      the current task, even if the current task is not the
      group leader.
      
      But local process wide clock samples are allowed if PID == 0
      (current task) even if the current task is not the group leader.
      So in the end this should be no big deal as this actually harmonize
      the behaviour when the remote sample is actually a local one.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Kosaki Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      33ab0fec
    • F
      posix-timers: Remove useless clock sample on timers cleanup · af82eb3c
      Frederic Weisbecker 提交于
      a0b2062b
      ("posix_timers: fix racy timer delta caching on task exit") forgot
      to remove the arguments used for timer caching.
      
      Fix this leftover.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Kosaki Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      af82eb3c
    • F
      posix-timers: Remove dead task special case · a3222f88
      Frederic Weisbecker 提交于
      Now that we've removed all the optimizations that could
      result in NULL timer's targets, we can remove all the
      associated special case handling.
      
      Also add some warnings on NULL targets to spot any possible
      leftover.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Kosaki Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      a3222f88
    • F
      posix-timers: Cleanup reaped target handling · e26d70d2
      Frederic Weisbecker 提交于
      When a timer's target is seen to be buried, for example on calls
      to timer_gettime(), the posix cpu timers code behaves a bit
      like a garbage collector and releases early the reference to the
      task.
      
      Then again, this optimization complicates the code for no much
      value: it's up to the user to release the timer and its associated
      ressources by calling timer_delete() after it buries the target
      tasks.
      
      Remove this to simplify the code.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Kosaki Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      e26d70d2
    • F
      posix-timers: Remove dead process posix cpu timers caching · d430b917
      Frederic Weisbecker 提交于
      Now that we removed dead thread posix cpu timers caching,
      lets remove the dead process wide version. This caching
      is similar to the per thread version but it should be even
      more rare:
      
      * If the process id dead, we are not reading its timers
      status from a thread belonging to its group since they
      are all dead. So this caching only concern remote process
      timers reads. Now posix cpu timers using itimers or timer_settime()
      can't do remote process timers anyway so it's not even clear if there
      is actually a user for this caching.
      
      * Unlike per thread timers caching, this only applies to
      zombies targets. Buried targets' process wide timers return
      0 values. But then again, timer_gettime() can't read remote
      process timers, so if the process is dead, there can't be
      any reader left anyway.
      
      Then again this caching seem to complicate the code for
      corner cases that are probably not worth it. So lets get
      rid of it.
      
      Also remove the sample snapshot on dying process timer
      that is now useless, as suggested by Kosaki.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Kosaki Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      d430b917
    • F
      posix-timers: Remove dead thread posix cpu timers caching · 724a3713
      Frederic Weisbecker 提交于
      When a task is exiting or has exited, its posix cpu timers
      don't tick anymore and won't elapse further. It's too late
      for them to expire.
      
      So any further call to timer_gettime() on these timers will
      return the same remaining expiry time.
      
      The current code optimize this by caching the remaining delta
      and storing it where we use to save the absolute expiration time.
      This way, the future calls to timer_gettime() won't need to
      compute the difference between the absolute expiration time and
      the current time anymore.
      
      Now this optimization doesn't seem to bring much value. Computing
      the timer remaining delta is not very costly. Fetching the timer
      value OTOH can be costly in two ways:
      
      * CPUCLOCK_SCHED read requires to lock the target's rq. But some
      optimizations are on the way to make task_sched_runtime() not holding
      the rq lock of a non-running target.
      
      * CPUCLOCK_VIRT/CPUCLOCK_PROF read simply consist in fetching
      current->utime/current->stime except when the system uses full
      dynticks cputime accounting. The latter requires a per task lock
      in order to correctly compute user and system time. But once the
      target is dead, this lock shouldn't be contended anyway.
      
      All in one this caching doesn't seem to be justified.
      Given that it complicates the code significantly for
      few wins, let's remove it on single thread timers.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Kosaki Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      724a3713
  10. 03 12月, 2013 2 次提交
    • F
      posix-timers: Fix full dynticks CPUs kick on timer rescheduling · c925077c
      Frederic Weisbecker 提交于
      A posix CPU timer can be rearmed while it is firing or after it is
      notified with a signal. This can happen for example with timers that
      were set with a non zero interval in timer_settime().
      
      This rearming can happen in two places:
      
      1) On timer firing time, which happens on the target's tick. If the timer
      can't trigger a signal because it is ignored, it reschedules itself
      to honour the timer interval.
      
      2) On signal handling from the timer's notification target. This one
      can be a different task than the timer's target itself. Once the
      signal is notified, the notification target rearms the timer, again
      to honour the timer interval.
      
      When a timer is rearmed, we need to notify the full dynticks CPUs
      such that they restart their tick in case they are running tasks that
      may have a share in elapsing this timer.
      
      Now the 1st case above handles full dynticks CPUs with a call to
      posix_cpu_timer_kick_nohz() from the posix cpu timer firing code. But
      the second case ignores the fact that some CPUs may run non-idle tasks
      with their tick off. As a result, when a timer is resheduled after its signal
      notification, the full dynticks CPUs may completely ignore it and not
      tick on the timer as expected
      
      This patch fixes this bug by handling both cases in one. All we need
      is to move the kick to the rearming common code in posix_cpu_timer_schedule().
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Olivier Langlois <olivier@olivierlanglois.net>
      c925077c
    • F
      posix-timers: Spare workqueue if there is no full dynticks CPU to kick · d4283c65
      Frederic Weisbecker 提交于
      After a posix cpu timer is set, a workqueue is scheduled in order to
      kick the full dynticks CPUs and let them restart their tick if
      necessary in case the task they are running is concerned by the
      new timer.
      
      This kick is implemented by way of IPIs, which require interrupts
      to be enabled, hence the need for a workqueue to raise them because
      the posix cpu timer set path has interrupts disabled.
      
      Now if there is no full dynticks CPU on the system, the workqueue is
      still scheduled but it simply won't send any IPI and return immediately.
      
      So lets spare that worqueue when it is not needed.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      d4283c65
  11. 03 7月, 2013 5 次提交
    • F
      posix_timers: fix racy timer delta caching on task exit · a0b2062b
      Frederic Weisbecker 提交于
      When a task exits, we perform a caching of the remaining cputime delta
      before expiring of its timers.
      
      This is done from the following places:
      
      * When the task is reaped. We iterate through its list of
        posix cpu timers and store the remaining timer delta to
        the timer struct instead of the absolute value.
        (See posix_cpu_timers_exit() / posix_cpu_timers_exit_group() )
      
      * When we call posix_cpu_timer_get() or posix_cpu_timer_schedule().
        If the timer's task is considered dying when watched from these
        places, the same conversion from absolute to relative expiry time
        is performed. Then the given task's reference is released.
        (See clear_dead_task() ).
      
      The relevance of this caching is questionable but this is another
      and deeper debate.
      
      The big issue here is that these two sources of caching don't mix
      up very well together.
      
      More specifically, the caching can easily be done twice, resulting
      in a wrong delta as it gets spuriously substracted a second time by
      the elapsed clock. This can happen in the following scenario:
      
      1) The task exits and gets reaped: we call posix_cpu_timers_exit()
         and the absolute timer expiry values are converted to a relative
         delta.
      
      2) timer_gettime() -> posix_cpu_timer_get() is called and relies on
         clear_dead_task() because  tsk->exit_state == EXIT_DEAD.
         The delta gets substracted again by the elapsed clock and we return
         a wrong result.
      
      To fix this, just remove the caching done on task reaping time.  It
      doesn't bring much value on its own.  The caching done from
      posix_cpu_timer_get/schedule is enough.
      
      And it would also be hard to get it really right: we could make it put and
      clear the target task in the timer struct so that readers know if they are
      dealing with a relative cached of absolute value.  But it would be racy.
      The only safe way to do it would be to lock the itimer->it_lock so that we
      know nobody reads the cputime expiry value while we modify it and its
      target task reference.  Doing so would involve some funny workarounds to
      avoid circular lock against the sighand lock.  There is just no reason to
      maintain this.
      
      The user visible effect of this patch can be observed by running the
      following code: it creates a subthread that launches a posix cputimer
      which expires after 10 seconds. But then the subthread only busy loops for 2
      seconds and exits. The parent reaps the subthread and read the timer value.
      Its expected value should the be the initial timer's expiration value
      minus the cputime elapsed in the subthread. Roughly 10 - 2 = 8 seconds:
      
      	#include <sys/time.h>
      	#include <stdio.h>
      	#include <unistd.h>
      	#include <time.h>
      	#include <pthread.h>
      
      	static timer_t id;
      	static struct itimerspec val = { .it_value.tv_sec = 10, }, new;
      
      	static void *thread(void *unused)
      	{
      		int err;
      		struct timeval start, end, diff;
      
      		timer_create(CLOCK_THREAD_CPUTIME_ID, NULL, &id);
      		if (err < 0) {
      			perror("Can't create timer\n");
      			return NULL;
      		}
      
      		/* Arm 10 sec timer */
      		err = timer_settime(id, 0, &val, NULL);
      		if (err < 0) {
      			perror("Can't set timer\n");
      			return NULL;
      		}
      
      		/* Exit after 2 seconds of execution */
      		gettimeofday(&start, NULL);
      	        do {
      			gettimeofday(&end, NULL);
      			timersub(&end, &start, &diff);
      		} while (diff.tv_sec < 2);
      
      		return NULL;
      	}
      
      	int main(int argc, char **argv)
      	{
      		pthread_t pthread;
      		int err;
      
      		err = pthread_create(&pthread, NULL, thread, NULL);
      		if (err) {
      			perror("Can't create thread\n");
      			return -1;
      		}
      		pthread_join(pthread, NULL);
      		/* Just wait a little bit to make sure the child got reaped */
      		sleep(1);
      		err = timer_gettime(id, &new);
      		if (err)
      			perror("Can't get timer value\n");
      		printf("%d %ld\n", new.it_value.tv_sec, new.it_value.tv_nsec);
      
      		return 0;
      	}
      
      Before the patch:
      
             $ ./posix_cpu_timers
             6 2278074
      
      After the patch:
      
            $ ./posix_cpu_timers
            8 1158766
      
      Before the patch, the elapsed time got two more seconds spuriously accounted.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Stanislaw Gruszka <sgruszka@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
      Cc: Olivier Langlois <olivier@trillion01.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      a0b2062b
    • F
      posix-timers: correctly get dying task time sample in posix_cpu_timer_schedule() · 76cdcdd9
      Frederic Weisbecker 提交于
      In order to re-arm a timer after it fired, we take a sample of the current
      process or thread cputime.
      
      If the task is dying though, we don't arm anything but we cache the
      remaining timer expiration delta for further reads.
      
      Something similar is performed in posix_cpu_timer_get() but here we forget
      to take the process wide cputime sample before caching it.
      
      As a result we are storing random stack content, leading every further
      reads of that timer to return junk values.
      
      Fix this by taking the appropriate sample in the case of process wide
      timers.
      
      This probably doesn't matter much in practice because, at this stage, the
      thread is the last one in the group and we reached exit_notify().  This
      implies that we called exit_itimers() and there should be no more timers
      to handle for that task.
      
      So this is likely dead code anyway but let's fix the current logic
      and the warning that came along:
      
          kernel/posix-cpu-timers.c: In function 'posix_cpu_timer_schedule':
          kernel/posix-cpu-timers.c:1127: warning: 'now' may be used uninitialized in this function
      
      Then we can start to think further about cleaning up that code.
      Reported-by: NAndrew Morton <akpm@linux-foundation.org>
      Reported-by: NChen Gang <gang.chen@asianux.com>
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Stanislaw Gruszka <sgruszka@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Chen Gang <gang.chen@asianux.com>
      Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
      Cc: Olivier Langlois <olivier@trillion01.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      76cdcdd9
    • F
      posix_cpu_timers: consolidate expired timers check · 2473f3e7
      Frederic Weisbecker 提交于
      Consolidate the common code amongst per thread and per process timers list
      on tick time.
      
      List traversal, expiry check and subsequent updates can be shared in a
      common helper.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Stanislaw Gruszka <sgruszka@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
      Cc: Olivier Langlois <olivier@trillion01.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      2473f3e7
    • F
      posix_cpu_timers: consolidate timer list cleanups · 1a7fa510
      Frederic Weisbecker 提交于
      Cleaning up the posix cpu timers on task exit shares some common code
      among timer list types, most notably the list traversal and expiry time
      update.
      
      Unify this in a common helper.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Stanislaw Gruszka <sgruszka@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
      Cc: Olivier Langlois <olivier@trillion01.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      1a7fa510
    • F
      posix_cpu_timer: consolidate expiry time type · 55ccb616
      Frederic Weisbecker 提交于
      The posix cpu timer expiry time is stored in a union of two types: a 64
      bits field if we rely on scheduler precise accounting, or a cputime_t if
      we rely on jiffies.
      
      This results in quite some duplicate code and special cases to handle the
      two types.
      
      Just unify this into a single 64 bits field.  cputime_t can always fit
      into it.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Stanislaw Gruszka <sgruszka@redhat.com>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com>
      Cc: Olivier Langlois <olivier@trillion01.com>
      Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
      55ccb616
  12. 23 4月, 2013 1 次提交
    • F
      posix_timers: Fix pre-condition to stop the tick on full dynticks · 6ac29178
      Frederic Weisbecker 提交于
      The test that checks if a CPU can stop its tick from posix CPU
      timers angle was mistakenly inverted.
      
      What we want is to prevent the tick from being stopped as long
      as the current CPU's task runs a posix CPU timer.
      
      Fix this.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Chris Metcalf <cmetcalf@tilera.com>
      Cc: Christoph Lameter <cl@linux.com>
      Cc: Geoff Levand <geoff@infradead.org>
      Cc: Gilad Ben Yossef <gilad@benyossef.com>
      Cc: Hakan Akkan <hakanakkan@gmail.com>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Kevin Hilman <khilman@linaro.org>
      Cc: Li Zhong <zhong@linux.vnet.ibm.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      6ac29178
  13. 19 4月, 2013 2 次提交
    • F
      posix_timers: New API to prevent from stopping the tick when timers are running · 555347f6
      Frederic Weisbecker 提交于
      Bring a new helper that the full dynticks infrastructure can
      call in order to know if it can safely stop the tick from
      the posix cpu timers subsystem point of view.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Chris Metcalf <cmetcalf@tilera.com>
      Cc: Christoph Lameter <cl@linux.com>
      Cc: Geoff Levand <geoff@infradead.org>
      Cc: Gilad Ben Yossef <gilad@benyossef.com>
      Cc: Hakan Akkan <hakanakkan@gmail.com>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Kevin Hilman <khilman@linaro.org>
      Cc: Li Zhong <zhong@linux.vnet.ibm.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      555347f6
    • F
      posix_timers: Kick full dynticks CPUs when a posix cpu timer is armed · a8572160
      Frederic Weisbecker 提交于
      Kick the full dynticks CPUs when a posix cpu timer is enqueued by
      way of a standard call to posix_cpu_timer_set() or set_process_cpu_timer().
      This also include rescheduled firing timers.
      
      This way they can re-evaluate the state of (and possibly restart) their
      tick against the new expiry modification.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Chris Metcalf <cmetcalf@tilera.com>
      Cc: Christoph Lameter <cl@linux.com>
      Cc: Geoff Levand <geoff@infradead.org>
      Cc: Gilad Ben Yossef <gilad@benyossef.com>
      Cc: Hakan Akkan <hakanakkan@gmail.com>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Kevin Hilman <khilman@linaro.org>
      Cc: Li Zhong <zhong@linux.vnet.ibm.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      a8572160
  14. 15 2月, 2013 1 次提交
  15. 28 1月, 2013 1 次提交
    • F
      cputime: Use accessors to read task cputime stats · 6fac4829
      Frederic Weisbecker 提交于
      This is in preparation for the full dynticks feature. While
      remotely reading the cputime of a task running in a full
      dynticks CPU, we'll need to do some extra-computation. This
      way we can account the time it spent tickless in userspace
      since its last cputime snapshot.
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Li Zhong <zhong@linux.vnet.ibm.com>
      Cc: Namhyung Kim <namhyung.kim@lge.com>
      Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      6fac4829
  16. 17 12月, 2012 1 次提交
  17. 29 11月, 2012 1 次提交
  18. 15 12月, 2011 1 次提交
  19. 18 10月, 2011 1 次提交
    • P
      cputimer: Cure lock inversion · bcd5cff7
      Peter Zijlstra 提交于
      There's a lock inversion between the cputimer->lock and rq->lock;
      notably the two callchains involved are:
      
       update_rlimit_cpu()
         sighand->siglock
         set_process_cpu_timer()
           cpu_timer_sample_group()
             thread_group_cputimer()
               cputimer->lock
               thread_group_cputime()
                 task_sched_runtime()
                   ->pi_lock
                   rq->lock
      
       scheduler_tick()
         rq->lock
         task_tick_fair()
           update_curr()
             account_group_exec()
               cputimer->lock
      
      Where the first one is enabling a CLOCK_PROCESS_CPUTIME_ID timer, and
      the second one is keeping up-to-date.
      
      This problem was introduced by e8abccb7 ("posix-cpu-timers: Cure
      SMP accounting oddities").
      
      Cure the problem by removing the cputimer->lock and rq->lock nesting,
      this leaves concurrent enablers doing duplicate work, but the time
      wasted should be on the same order otherwise wasted spinning on the
      lock and the greater-than assignment filter should ensure we preserve
      monotonicity.
      Reported-by: NDave Jones <davej@redhat.com>
      Reported-by: NSimon Kirby <sim@hostway.ca>
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: stable@kernel.org
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
      Link: http://lkml.kernel.org/r/1318928713.21167.4.camel@twinsSigned-off-by: NThomas Gleixner <tglx@linutronix.de>
      bcd5cff7
  20. 30 9月, 2011 1 次提交
    • P
      posix-cpu-timers: Cure SMP wobbles · d670ec13
      Peter Zijlstra 提交于
      David reported:
      
        Attached below is a watered-down version of rt/tst-cpuclock2.c from
        GLIBC.  Just build it with "gcc -o test test.c -lpthread -lrt" or
        similar.
      
        Run it several times, and you will see cases where the main thread
        will measure a process clock difference before and after the nanosleep
        which is smaller than the cpu-burner thread's individual thread clock
        difference.  This doesn't make any sense since the cpu-burner thread
        is part of the top-level process's thread group.
      
        I've reproduced this on both x86-64 and sparc64 (using both 32-bit and
        64-bit binaries).
      
        For example:
      
        [davem@boricha build-x86_64-linux]$ ./test
        process: before(0.001221967) after(0.498624371) diff(497402404)
        thread:  before(0.000081692) after(0.498316431) diff(498234739)
        self:    before(0.001223521) after(0.001240219) diff(16698)
        [davem@boricha build-x86_64-linux]$ 
      
        The diff of 'process' should always be >= the diff of 'thread'.
      
        I make sure to wrap the 'thread' clock measurements the most tightly
        around the nanosleep() call, and that the 'process' clock measurements
        are the outer-most ones.
      
        ---
        #include <unistd.h>
        #include <stdio.h>
        #include <stdlib.h>
        #include <time.h>
        #include <fcntl.h>
        #include <string.h>
        #include <errno.h>
        #include <pthread.h>
      
        static pthread_barrier_t barrier;
      
        static void *chew_cpu(void *arg)
        {
      	  pthread_barrier_wait(&barrier);
      	  while (1)
      		  __asm__ __volatile__("" : : : "memory");
      	  return NULL;
        }
      
        int main(void)
        {
      	  clockid_t process_clock, my_thread_clock, th_clock;
      	  struct timespec process_before, process_after;
      	  struct timespec me_before, me_after;
      	  struct timespec th_before, th_after;
      	  struct timespec sleeptime;
      	  unsigned long diff;
      	  pthread_t th;
      	  int err;
      
      	  err = clock_getcpuclockid(0, &process_clock);
      	  if (err)
      		  return 1;
      
      	  err = pthread_getcpuclockid(pthread_self(), &my_thread_clock);
      	  if (err)
      		  return 1;
      
      	  pthread_barrier_init(&barrier, NULL, 2);
      	  err = pthread_create(&th, NULL, chew_cpu, NULL);
      	  if (err)
      		  return 1;
      
      	  err = pthread_getcpuclockid(th, &th_clock);
      	  if (err)
      		  return 1;
      
      	  pthread_barrier_wait(&barrier);
      
      	  err = clock_gettime(process_clock, &process_before);
      	  if (err)
      		  return 1;
      
      	  err = clock_gettime(my_thread_clock, &me_before);
      	  if (err)
      		  return 1;
      
      	  err = clock_gettime(th_clock, &th_before);
      	  if (err)
      		  return 1;
      
      	  sleeptime.tv_sec = 0;
      	  sleeptime.tv_nsec = 500000000;
      	  nanosleep(&sleeptime, NULL);
      
      	  err = clock_gettime(th_clock, &th_after);
      	  if (err)
      		  return 1;
      
      	  err = clock_gettime(my_thread_clock, &me_after);
      	  if (err)
      		  return 1;
      
      	  err = clock_gettime(process_clock, &process_after);
      	  if (err)
      		  return 1;
      
      	  diff = process_after.tv_nsec - process_before.tv_nsec;
      	  printf("process: before(%lu.%.9lu) after(%lu.%.9lu) diff(%lu)\n",
      		 process_before.tv_sec, process_before.tv_nsec,
      		 process_after.tv_sec, process_after.tv_nsec, diff);
      	  diff = th_after.tv_nsec - th_before.tv_nsec;
      	  printf("thread:  before(%lu.%.9lu) after(%lu.%.9lu) diff(%lu)\n",
      		 th_before.tv_sec, th_before.tv_nsec,
      		 th_after.tv_sec, th_after.tv_nsec, diff);
      	  diff = me_after.tv_nsec - me_before.tv_nsec;
      	  printf("self:    before(%lu.%.9lu) after(%lu.%.9lu) diff(%lu)\n",
      		 me_before.tv_sec, me_before.tv_nsec,
      		 me_after.tv_sec, me_after.tv_nsec, diff);
      
      	  return 0;
        }
      
      This is due to us using p->se.sum_exec_runtime in
      thread_group_cputime() where we iterate the thread group and sum all
      data. This does not take time since the last schedule operation (tick
      or otherwise) into account. We can cure this by using
      task_sched_runtime() at the cost of having to take locks.
      
      This also means we can (and must) do away with
      thread_group_sched_runtime() since the modified thread_group_cputime()
      is now more accurate and would deadlock when called from
      thread_group_sched_runtime().
      
      Aside of that it makes the function safe on 32 bit systems. The old
      code added t->se.sum_exec_runtime unprotected. sum_exec_runtime is a
      64bit value and could be changed on another cpu at the same time.
      Reported-by: NDavid Miller <davem@davemloft.net>
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: stable@kernel.org
      Link: http://lkml.kernel.org/r/1314874459.7945.22.camel@twinsTested-by: NDavid Miller <davem@davemloft.net>
      Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
      d670ec13