1. 07 4月, 2009 12 次提交
  2. 06 4月, 2009 28 次提交
    • P
      perf_counter: update mmap() counter read · 92f22a38
      Peter Zijlstra 提交于
      Paul noted that we don't need SMP barriers for the mmap() counter read
      because its always on the same cpu (otherwise you can't access the hw
      counter anyway).
      
      So remove the SMP barriers and replace them with regular compiler
      barriers.
      
      Further, update the comment to include a race free method of reading
      said hardware counter. The primary change is putting the pmc_read
      inside the seq-loop, otherwise we can still race and read rubbish.
      Noticed-by: NPaul Mackerras <paulus@samba.org>
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
      Orig-LKML-Reference: <20090402091319.577951445@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      92f22a38
    • P
      perf_counter: add more context information · 5872bdb8
      Peter Zijlstra 提交于
      Put in counts to tell which ips belong to what context.
      
        -----
         | |  hv
         | --
      nr | |  kernel
         | --
         | |  user
        -----
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
      Orig-LKML-Reference: <20090402091319.493101305@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      5872bdb8
    • P
      perf_counter: per event wakeups · c457810a
      Peter Zijlstra 提交于
      By request, provide a way to request a wakeup every 'n' events instead
      of every page of output.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
      Orig-LKML-Reference: <20090402091319.323309784@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      c457810a
    • P
      perf_counter: move the event overflow output bits to record_type · 8a057d84
      Peter Zijlstra 提交于
      Per suggestion from Paul, move the event overflow bits to record_type
      and sanitize the enums a bit.
      
      Breaks the ABI -- again ;-)
      Suggested-by: NPaul Mackerras <paulus@samba.org>
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
      Orig-LKML-Reference: <20090402091319.151921176@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      8a057d84
    • P
      perf_counter: provide generic callchain bits · 394ee076
      Peter Zijlstra 提交于
      Provide the generic callchain support bits. If hw_event->callchain is
      set the arch specific perf_callchain() function is called upon to
      provide a perf_callchain_entry structure filled with the current
      callchain.
      
      If it does so, it is added to the overflow output event.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Acked-by: NPaul Mackerras <paulus@samba.org>
      Orig-LKML-Reference: <20090330171024.254266860@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      394ee076
    • P
      perf_counter: re-arrange the perf_event_type · 5ed00415
      Peter Zijlstra 提交于
      Breaks ABI yet again :-)
      
      Change the event type so that [0, 2^31-1] are regular event types, but
      [2^31, 2^32-1] forms a bitmask for overflow events.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Acked-by: NPaul Mackerras <paulus@samba.org>
      Orig-LKML-Reference: <20090330171024.047961770@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      5ed00415
    • P
      perf_counter: small cleanup of the output routines · 78d613eb
      Peter Zijlstra 提交于
      Move the nmi argument to the _begin() function, so that _end() only needs the
      handle. This allows the _begin() function to generate a wakeup on event loss.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Acked-by: NPaul Mackerras <paulus@samba.org>
      Orig-LKML-Reference: <20090330171023.959404268@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      78d613eb
    • P
      perf_counter: make it possible for hw_perf_counter_init to return error codes · d5d2bc0d
      Paul Mackerras 提交于
      Impact: better error reporting
      
      At present, if hw_perf_counter_init encounters an error, all it can do
      is return NULL, which causes sys_perf_counter_open to return an EINVAL
      error to userspace.  This isn't very informative for userspace; it means
      that userspace can't tell the difference between "sorry, oprofile is
      already using the PMU" and "we don't support this CPU" and "this CPU
      doesn't support the requested generic hardware event".
      
      This commit uses the PTR_ERR/ERR_PTR/IS_ERR set of macros to let
      hw_perf_counter_init return an error code on error rather than just NULL
      if it wishes.  If it does so, that error code will be returned from
      sys_perf_counter_open to userspace.  If it returns NULL, an EINVAL
      error will be returned to userspace, as before.
      
      This also adapts the powerpc hw_perf_counter_init to make use of this
      to return ENXIO, EINVAL, EBUSY, or EOPNOTSUPP as appropriate.  It would
      be good to add extra error numbers in future to allow userspace to
      distinguish the various errors that are currently reported as EINVAL,
      i.e. irq_period < 0, too many events in a group, conflict between
      exclude_* settings in a group, and PMU resource conflict in a group.
      
      [ v2: fix a bug pointed out by Corey Ashford where error returns from
            hw_perf_counter_init were not handled correctly in the case of
            raw hardware events.]
      Signed-off-by: NPaul Mackerras <paulus@samba.org>
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Orig-LKML-Reference: <20090330171023.682428180@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      d5d2bc0d
    • P
      perf_counter: executable mmap() information · 0a4a9391
      Peter Zijlstra 提交于
      Currently the profiling information returns userspace IPs but no way
      to correlate them to userspace code. Userspace could look into
      /proc/$pid/maps but that might not be current or even present anymore
      at the time of analyzing the IPs.
      
      Therefore provide means to track the mmap information and provide it
      in the output stream.
      
      XXX: only covers mmap()/munmap(), mremap() and mprotect() are missing.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Acked-by: NPaul Mackerras <paulus@samba.org>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Orig-LKML-Reference: <20090330171023.417259499@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      0a4a9391
    • P
      perf_counter: fix update_userpage() · 38ff667b
      Peter Zijlstra 提交于
      It just occured to me it is possible to have multiple contending
      updates of the userpage (mmap information vs overflow vs counter).
      This would break the seqlock logic.
      
      It appear the arch code uses this from NMI context, so we cannot
      possibly serialize its use, therefore separate the data_head update
      from it and let it return to its original use.
      
      The arch code needs to make sure there are no contending callers by
      disabling the counter before using it -- powerpc appears to do this
      nicely.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Acked-by: NPaul Mackerras <paulus@samba.org>
      Orig-LKML-Reference: <20090330171023.241410660@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      38ff667b
    • P
      perf_counter: unify and fix delayed counter wakeup · 925d519a
      Peter Zijlstra 提交于
      While going over the wakeup code I noticed delayed wakeups only work
      for hardware counters but basically all software counters rely on
      them.
      
      This patch unifies and generalizes the delayed wakeup to fix this
      issue.
      
      Since we're dealing with NMI context bits here, use a cmpxchg() based
      single link list implementation to track counters that have pending
      wakeups.
      
      [ This should really be generic code for delayed wakeups, but since we
        cannot use cmpxchg()/xchg() in generic code, I've let it live in the
        perf_counter code. -- Eric Dumazet could use it to aggregate the
        network wakeups. ]
      
      Furthermore, the x86 method of using TIF flags was flawed in that its
      quite possible to end up setting the bit on the idle task, loosing the
      wakeup.
      
      The powerpc method uses per-cpu storage and does appear to be
      sufficient.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Acked-by: NPaul Mackerras <paulus@samba.org>
      Orig-LKML-Reference: <20090330171023.153932974@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      925d519a
    • P
      perf_counter: record time running and time enabled for each counter · 53cfbf59
      Paul Mackerras 提交于
      Impact: new functionality
      
      Currently, if there are more counters enabled than can fit on the CPU,
      the kernel will multiplex the counters on to the hardware using
      round-robin scheduling.  That isn't too bad for sampling counters, but
      for counting counters it means that the value read from a counter
      represents some unknown fraction of the true count of events that
      occurred while the counter was enabled.
      
      This remedies the situation by keeping track of how long each counter
      is enabled for, and how long it is actually on the cpu and counting
      events.  These times are recorded in nanoseconds using the task clock
      for per-task counters and the cpu clock for per-cpu counters.
      
      These values can be supplied to userspace on a read from the counter.
      Userspace requests that they be supplied after the counter value by
      setting the PERF_FORMAT_TOTAL_TIME_ENABLED and/or
      PERF_FORMAT_TOTAL_TIME_RUNNING bits in the hw_event.read_format field
      when creating the counter.  (There is no way to change the read format
      after the counter is created, though it would be possible to add some
      way to do that.)
      
      Using this information it is possible for userspace to scale the count
      it reads from the counter to get an estimate of the true count:
      
      true_count_estimate = count * total_time_enabled / total_time_running
      
      This also lets userspace detect the situation where the counter never
      got to go on the cpu: total_time_running == 0.
      
      This functionality has been requested by the PAPI developers, and will
      be generally needed for interpreting the count values from counting
      counters correctly.
      
      In the implementation, this keeps 5 time values (in nanoseconds) for
      each counter: total_time_enabled and total_time_running are used when
      the counter is in state OFF or ERROR and for reporting back to
      userspace.  When the counter is in state INACTIVE or ACTIVE, it is the
      tstamp_enabled, tstamp_running and tstamp_stopped values that are
      relevant, and total_time_enabled and total_time_running are determined
      from them.  (tstamp_stopped is only used in INACTIVE state.)  The
      reason for doing it like this is that it means that only counters
      being enabled or disabled at sched-in and sched-out time need to be
      updated.  There are no new loops that iterate over all counters to
      update total_time_enabled or total_time_running.
      
      This also keeps separate child_total_time_running and
      child_total_time_enabled fields that get added in when reporting the
      totals to userspace.  They are separate fields so that they can be
      atomic.  We don't want to use atomics for total_time_running,
      total_time_enabled etc., because then we would have to use atomic
      sequences to update them, which are slower than regular arithmetic and
      memory accesses.
      
      It is possible to measure total_time_running by adding a task_clock
      counter to each group of counters, and total_time_enabled can be
      measured approximately with a top-level task_clock counter (though
      inaccuracies will creep in if you need to disable and enable groups
      since it is not possible in general to disable/enable the top-level
      task_clock counter simultaneously with another group).  However, that
      adds extra overhead - I measured around 15% increase in the context
      switch latency reported by lat_ctx (from lmbench) when a task_clock
      counter was added to each of 2 groups, and around 25% increase when a
      task_clock counter was added to each of 4 groups.  (In both cases a
      top-level task-clock counter was also added.)
      
      In contrast, the code added in this commit gives better information
      with no overhead that I could measure (in fact in some cases I
      measured lower times with this code, but the differences were all less
      than one standard deviation).
      
      [ v2: address review comments by Andrew Morton. ]
      Signed-off-by: NPaul Mackerras <paulus@samba.org>
      Acked-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Orig-LKML-Reference: <18890.6578.728637.139402@cargo.ozlabs.ibm.com>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      53cfbf59
    • P
      perf_counter: allow and require one-page mmap on counting counters · 7730d865
      Peter Zijlstra 提交于
      A brainfart stopped single page mmap()s working. The rest of the code
      should be perfectly fine with not having any data pages.
      Reported-by: NPaul Mackerras <paulus@samba.org>
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Orig-LKML-Reference: <1237981712.7972.812.camel@twins>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      7730d865
    • P
      perf_counter: optionally provide the pid/tid of the sampled task · ea5d20cf
      Peter Zijlstra 提交于
      Allow cpu wide counters to profile userspace by providing what process
      the sample belongs to.
      
      This raises the first issue with the output type, lots of these
      options: group, tid, callchain, etc.. are non-exclusive and could be
      combined, suggesting a bitfield.
      
      However, things like the mmap() data stream doesn't fit in that.
      
      How to split the type field...
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Arjan van de Ven <arjan@infradead.org>
      Cc: Wu Fengguang <fengguang.wu@intel.com>
      Orig-LKML-Reference: <20090325113317.013775235@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      ea5d20cf
    • P
      perf_counter: sanity check on the output API · 63e35b25
      Peter Zijlstra 提交于
      Ensure we never write more than we said we would.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Arjan van de Ven <arjan@infradead.org>
      Cc: Wu Fengguang <fengguang.wu@intel.com>
      Orig-LKML-Reference: <20090325113316.921433024@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      63e35b25
    • P
      perf_counter: output objects · 5c148194
      Peter Zijlstra 提交于
      Provide a {type,size} header for each output entry.
      
      This should provide extensible output, and the ability to mix multiple streams.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Arjan van de Ven <arjan@infradead.org>
      Cc: Wu Fengguang <fengguang.wu@intel.com>
      Orig-LKML-Reference: <20090325113316.831607932@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      5c148194
    • P
      perf_counter: more elaborate write API · b9cacc7b
      Peter Zijlstra 提交于
      Provide a begin, copy, end interface to the output buffer.
      
      begin() reserves the space,
       copy() copies the data over, considering page boundaries,
        end() finalizes the event and does the wakeup.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Arjan van de Ven <arjan@infradead.org>
      Cc: Wu Fengguang <fengguang.wu@intel.com>
      Orig-LKML-Reference: <20090325113316.740550870@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      b9cacc7b
    • P
      perf_counter: fix perf_poll() · c7138f37
      Peter Zijlstra 提交于
      Impact: fix kerneltop 100% CPU usage
      
      Only return a poll event when there's actually been one, poll_wait()
      doesn't actually wait for the waitq you pass it, it only enqueues
      you on it.
      
      Only once all FDs have been iterated and none of thm returned a
      poll-event will it schedule().
      
      Also make it return POLL_HUP when there's not mmap() area to read from.
      
      Further, fix a silly bug in the write code.
      Reported-by: NMike Galbraith <efault@gmx.de>
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arjan van de Ven <arjan@infradead.org>
      Orig-LKML-Reference: <1237897096.24918.181.camel@twins>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      c7138f37
    • P
      perf_counter: new output ABI - part 1 · 7b732a75
      Peter Zijlstra 提交于
      Impact: Rework the perfcounter output ABI
      
      use sys_read() only for instant data and provide mmap() output for all
      async overflow data.
      
      The first mmap() determines the size of the output buffer. The mmap()
      size must be a PAGE_SIZE multiple of 1+pages, where pages must be a
      power of 2 or 0. Further mmap()s of the same fd must have the same
      size. Once all maps are gone, you can again mmap() with a new size.
      
      In case of 0 extra pages there is no data output and the first page
      only contains meta data.
      
      When there are data pages, a poll() event will be generated for each
      full page of data. Furthermore, the output is circular. This means
      that although 1 page is a valid configuration, its useless, since
      we'll start overwriting it the instant we report a full page.
      
      Future work will focus on the output format (currently maintained)
      where we'll likey want each entry denoted by a header which includes a
      type and length.
      
      Further future work will allow to splice() the fd, also containing the
      async overflow data -- splice() would be mutually exclusive with
      mmap() of the data.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Orig-LKML-Reference: <20090323172417.470536358@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      7b732a75
    • P
      perf_counter: add an mmap method to allow userspace to read hardware counters · 37d81828
      Paul Mackerras 提交于
      Impact: new feature giving performance improvement
      
      This adds the ability for userspace to do an mmap on a hardware counter
      fd and get access to a read-only page that contains the information
      needed to translate a hardware counter value to the full 64-bit
      counter value that would be returned by a read on the fd.  This is
      useful on architectures that allow user programs to read the hardware
      counters, such as PowerPC.
      
      The mmap will only succeed if the counter is a hardware counter
      monitoring the current process.
      
      On my quad 2.5GHz PowerPC 970MP machine, userspace can read a counter
      and translate it to the full 64-bit value in about 30ns using the
      mmapped page, compared to about 830ns for the read syscall on the
      counter, so this does give a significant performance improvement.
      Signed-off-by: NPaul Mackerras <paulus@samba.org>
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Orig-LKML-Reference: <20090323172417.297057964@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      37d81828
    • P
      perf_counter: avoid recursion · 96f6d444
      Peter Zijlstra 提交于
      Tracepoint events like lock_acquire and software counters like
      pagefaults can recurse into the perf counter code again, avoid that.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Orig-LKML-Reference: <20090323172417.152096433@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      96f6d444
    • P
      perf_counter: remove the event config bitfields · f4a2deb4
      Peter Zijlstra 提交于
      Since the bitfields turned into a bit of a mess, remove them and rely on
      good old masks.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Orig-LKML-Reference: <20090323172417.059499915@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      f4a2deb4
    • P
      perf_counter: unify irq output code · 0322cd6e
      Peter Zijlstra 提交于
      Impact: cleanup
      
      Having 3 slightly different copies of the same code around does nobody
      any good. First step in revamping the output format.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Orig-LKML-Reference: <20090319194233.929962222@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      0322cd6e
    • P
      perf_counter: revamp syscall input ABI · b8e83514
      Peter Zijlstra 提交于
      Impact: modify ABI
      
      The hardware/software classification in hw_event->type became a little
      strained due to the addition of tracepoint tracing.
      
      Instead split up the field and provide a type field to explicitly specify
      the counter type, while using the event_id field to specify which event to
      use.
      
      Raw counters still work as before, only the raw config now goes into
      raw_event.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Orig-LKML-Reference: <20090319194233.836807573@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      b8e83514
    • P
      perf_counter: hook up the tracepoint events · e077df4f
      Peter Zijlstra 提交于
      Impact: new perfcounters feature
      
      Enable usage of tracepoints as perf counter events.
      
      tracepoint event ids can be found in /debug/tracing/event/*/*/id
      and (for now) are represented as -65536+id in the type field.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Orig-LKML-Reference: <20090319194233.744044174@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      e077df4f
    • P
      perf_counter: fix up counter free paths · f1600952
      Peter Zijlstra 提交于
      Impact: fix crash during perfcounters use
      
      I found another counter free path, create a free_counter() call to
      accomodate generic tear-down.
      
      Fixes an RCU bug.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Orig-LKML-Reference: <20090319194233.652078652@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      f1600952
    • P
      perf_counter: generic context switch event · 4a0deca6
      Peter Zijlstra 提交于
      Impact: cleanup
      
      Use the generic software events for context switches.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Orig-LKML-Reference: <20090319194233.283522645@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      4a0deca6
    • P
      perf_counter: fix uninitialized usage of event_list · 01ef09d9
      Peter Zijlstra 提交于
      Impact: fix boot crash
      
      When doing the generic context switch event I ran into some early
      boot hangs, which were caused by inf func recursion (event, fault,
      event, fault).
      
      I eventually tracked it down to event_list not being initialized
      at the time of the first event. Fix this.
      Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Orig-LKML-Reference: <20090319194233.195392657@chello.nl>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      01ef09d9