1. 02 6月, 2009 1 次提交
    • S
      ftrace: add kernel command line function filtering · 2af15d6a
      Steven Rostedt 提交于
      When using ftrace=function on the command line to trace functions
      on boot up, one can not filter out functions that are commonly called.
      
      This patch adds two new ftrace command line commands.
      
        ftrace_notrace=function-list
        ftrace_filter=function-list
      
      Where function-list is a comma separated list of functions to filter.
      The ftrace_notrace will make the functions listed not be included
      in the function tracing, and ftrace_filter will only trace the functions
      listed.
      
      These two act the same as the debugfs/tracing/set_ftrace_notrace and
      debugfs/tracing/set_ftrace_filter respectively.
      
      The simple glob expressions that are allowed by the filter files can also
      be used by the command line interface.
      
      	ftrace_notrace=rcu*,*lock,*spin*
      
      Will not trace any function that starts with rcu, ends with lock, or has
      the word spin in it.
      
      Note, if the self tests are enabled, they may interfere with the filtering
      set by the command lines.
      Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>
      2af15d6a
  2. 18 5月, 2009 1 次提交
  3. 17 4月, 2009 1 次提交
    • S
      ftrace: use module notifier for function tracer · 93eb677d
      Steven Rostedt 提交于
      The hooks in the module code for the function tracer must be called
      before any of that module code runs. The function tracer hooks
      modify the module (replacing calls to mcount to nops). If the code
      is executed while the change occurs, then the CPU can take a GPF.
      
      To handle the above with a bit of paranoia, I originally implemented
      the hooks as calls directly from the module code.
      
      After examining the notifier calls, it looks as though the start up
      notify is called before any of the module's code is executed. This makes
      the use of the notify safe with ftrace.
      
      Only the startup notify is required to be "safe". The shutdown simply
      removes the entries from the ftrace function list, and does not modify
      any code.
      
      This change has another benefit. It removes a issue with a reverse dependency
      in the mutexes of ftrace_lock and module_mutex.
      
      [ Impact: fix lock dependency bug, cleanup ]
      
      Cc: Rusty Russell <rusty@rustcorp.com.au>
      Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>
      93eb677d
  4. 15 4月, 2009 1 次提交
    • S
      tracing/events: move trace point headers into include/trace/events · ad8d75ff
      Steven Rostedt 提交于
      Impact: clean up
      
      Create a sub directory in include/trace called events to keep the
      trace point headers in their own separate directory. Only headers that
      declare trace points should be defined in this directory.
      
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Neil Horman <nhorman@tuxdriver.com>
      Cc: Zhao Lei <zhaolei@cn.fujitsu.com>
      Cc: Eduard - Gabriel Munteanu <eduard.munteanu@linux360.ro>
      Cc: Pekka Enberg <penberg@cs.helsinki.fi>
      Signed-off-by: NSteven Rostedt <rostedt@goodmis.org>
      ad8d75ff
  5. 07 4月, 2009 2 次提交
  6. 01 4月, 2009 1 次提交
    • S
      function-graph: allow unregistering twice · 2aad1b76
      Steven Rostedt 提交于
      Impact: fix to permanent disabling of function graph tracer
      
      There should be nothing to prevent a tracer from unregistering a
      function graph callback more than once. This can simplify error paths.
      
      But currently, the counter does not account for mulitple unregistering
      of the function graph callback. If it happens, the function graph
      tracer will be permanently disabled.
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      2aad1b76
  7. 30 3月, 2009 1 次提交
  8. 26 3月, 2009 4 次提交
  9. 25 3月, 2009 5 次提交
    • S
      function-graph: add option to calculate graph time or not · a2a16d6a
      Steven Rostedt 提交于
      graph time is the time that a function is executing another function.
      Thus if function A calls B, if graph-time is set, then the time for
      A includes B. This is the default behavior. But if graph-time is off,
      then the time spent executing B is subtracted from A.
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      a2a16d6a
    • S
      tracing: make the function profiler per cpu · cafb168a
      Steven Rostedt 提交于
      Impact: speed enhancement
      
      By making the function profiler record in per cpu data we not only
      get better readings, avoid races, we also do not have to take any
      locks.
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      cafb168a
    • S
      tracing: adding function timings to function profiler · 0706f1c4
      Steven Rostedt 提交于
      If the function graph trace is enabled, the function profiler will
      use it to take the timing of the functions.
      
       cat /debug/tracing/trace_stat/functions
      
        Function                               Hit    Time
        --------                               ---    ----
        mwait_idle                             127    183028.4 us
        schedule                                26    151997.7 us
        __schedule                              31    151975.1 us
        sys_wait4                                2    74080.53 us
        do_wait                                  2    74077.80 us
        sys_newlstat                           138    39929.16 us
        do_path_lookup                         179    39845.79 us
        vfs_lstat_fd                           138    39761.97 us
        user_path_at                           153    39469.58 us
        path_walk                              179    39435.76 us
        __link_path_walk                       189    39143.73 us
      [...]
      
      Note the times are skewed due to the function graph tracer not taking
      into account schedules.
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      0706f1c4
    • S
      tracing: move function profiler data out of function struct · 493762fc
      Steven Rostedt 提交于
      Impact: reduce size of memory in function profiler
      
      The function profiler originally introduces its counters into the
      function records itself. There is 20 thousand different functions on
      a normal system, and that is adding 20 thousand counters for profiling
      event when not needed.
      
      A normal run of the profiler yields only a couple of thousand functions
      executed, depending on what is being profiled. This means we have around
      18 thousand useless counters.
      
      This patch rectifies this by moving the data out of the function
      records used by dynamic ftrace. Data is preallocated to hold the functions
      when the profiling begins. Checks are made during profiling to see if
      more recorcds should be allocated, and they are allocated if it is safe
      to do so.
      
      This also removes the dependency from using dynamic ftrace, and also
      removes the overhead by having it enabled.
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      493762fc
    • S
      tracing: add function profiler · bac429f0
      Steven Rostedt 提交于
      Impact: new profiling feature
      
      This patch adds a function profiler. In debugfs/tracing/ two new
      files are created.
      
        function_profile_enabled  - to enable or disable profiling
      
        trace_stat/functions   - the profiled functions.
      
      For example:
      
        echo 1 > /debugfs/tracing/function_profile_enabled
        ./hackbench 50
        echo 0 > /debugfs/tracing/function_profile_enabled
      
      yields:
      
        cat /debugfs/tracing/trace_stat/functions
      
        Function                               Hit
        --------                               ---
        _spin_lock                        10106442
        _spin_unlock                      10097492
        kfree                              6013704
        _spin_unlock_irqrestore            4423941
        _spin_lock_irqsave                 4406825
        __phys_addr                        4181686
        __slab_free                        4038222
        dput                               4030130
        path_put                           4023387
        unroll_tree_refs                   4019532
      [...]
      
      The most hit functions are listed first. Functions that are not
      hit are not listed.
      
      This feature depends on and uses dynamic function tracing. When the
      function profiling is disabled, no overhead occurs. But it still
      takes up around 300KB to hold the data, thus it is not recomended
      to keep it enabled for systems low on memory.
      
      When a '1' is echoed into the function_profile_enabled file, the
      counters for is function is reset back to zero. Thus you can see what
      functions are hit most by different programs.
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      bac429f0
  10. 24 3月, 2009 5 次提交
    • L
      tracing: use union for multi-usages field · ee000b7f
      Lai Jiangshan 提交于
      Impact: cleanup
      
      struct dyn_ftrace::ip has different usages in his lifecycle,
      we use union for it. And also for struct dyn_ftrace::flags.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Cc: Steven Rostedt <srostedt@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      LKML-Reference: <49C871BE.3080405@cn.fujitsu.com>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      ee000b7f
    • L
      ftrace: show virtual PID · cc59c9e8
      Lai Jiangshan 提交于
      Impact: fix PID output under namespaces
      
      When current namespace is not the global namespace,
      pid read from set_ftrace_pid is no correct.
      
       # ~/newpid_namespace_run bash
       # echo $$
       1
       # echo 1 > set_ftrace_pid
       # cat set_ftrace_pid
       3756
      
      Since we write virtual PID to set_ftrace_pid, we need get
      virtual PID when we read it.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Cc: Steven Rostedt <srostedt@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      LKML-Reference: <49C84D65.9050606@cn.fujitsu.com>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      cc59c9e8
    • S
      function-graph: add option for include sleep times · be6f164a
      Steven Rostedt 提交于
      Impact: give user a choice to show times spent while sleeping
      
      The user may want to see the time a function spent sleeping.
      This patch adds the trace option "sleep-time" to allow that.
      The "sleep-time" option is default on.
      
       echo sleep-time > /debug/tracing/trace_options
      
      produces:
      
       ------------------------------------------
       2)  avahi-d-3428  =>    <idle>-0
       ------------------------------------------
      
       2)               |      finish_task_switch() {
       2)   0.621 us    |        _spin_unlock_irq();
       2)   2.202 us    |      }
       2) ! 1002.197 us |    }
       2) ! 1003.521 us |  }
      
      where as,
      
       echo nosleep-time > /debug/tracing/trace_options
      
      produces:
      
       0)    <idle>-0    =>  yum-upd-3416
       ------------------------------------------
      
       0)               |              finish_task_switch() {
       0)   0.643 us    |                _spin_unlock_irq();
       0)   2.342 us    |              }
       0) + 41.302 us   |            }
       0) + 42.453 us   |          }
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      be6f164a
    • S
      function-graph: ignore times across schedule · 8aef2d28
      Steven Rostedt 提交于
      Impact: more accurate timings
      
      The current method of function graph tracing does not take into
      account the time spent when a task is not running. This shows functions
      that call schedule have increased costs:
      
       3) + 18.664 us   |      }
       ------------------------------------------
       3)    <idle>-0    =>  kblockd-123
       ------------------------------------------
      
       3)               |      finish_task_switch() {
       3)   1.441 us    |        _spin_unlock_irq();
       3)   3.966 us    |      }
       3) ! 2959.433 us |    }
       3) ! 2961.465 us |  }
      
      This patch uses the tracepoint in the scheduling context switch to
      account for time that has elapsed while a task is scheduled out.
      Now we see:
      
       ------------------------------------------
       3)    <idle>-0    =>  edac-po-1067
       ------------------------------------------
      
       3)               |      finish_task_switch() {
       3)   0.685 us    |        _spin_unlock_irq();
       3)   2.331 us    |      }
       3) + 41.439 us   |    }
       3) + 42.663 us   |  }
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      8aef2d28
    • S
      function-graph: prevent more than one tracer registering · 05ce5818
      Steven Rostedt 提交于
      Impact: prevent crash due to multiple function graph tracers
      
      The function graph tracer can currently only handle a single tracer
      being registered. If another tracer registers with the function
      graph tracer it can crash the system.
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      05ce5818
  11. 17 3月, 2009 1 次提交
  12. 13 3月, 2009 4 次提交
  13. 06 3月, 2009 2 次提交
  14. 21 2月, 2009 2 次提交
  15. 20 2月, 2009 1 次提交
    • F
      tracing/function-graph-tracer: make set_graph_function file support ftrace regex · f9349a8f
      Frederic Weisbecker 提交于
      Impact: trace only functions matching a pattern
      
      The set_graph_function file let one to trace only one or several
      chosen functions and follow all their code flow.
      
      Currently, only a constant function name is allowed so this patch
      allows the ftrace_regex functions:
      
      - matches all functions that end with "name":
        echo *name > set_graph_function
      
      - matches all functions that begin with "name":
        echo name* > set_graph_function
      
      - matches all functions that contains "name":
        echo *name* > set_graph_function
      
      Example:
      
      echo mutex* > set_graph_function
      
       0)               |  mutex_lock_nested() {
       0)   0.563 us    |    __might_sleep();
       0)   2.072 us    |  }
       0)               |  mutex_unlock() {
       0)   1.036 us    |    __mutex_unlock_slowpath();
       0)   2.433 us    |  }
       0)               |  mutex_unlock() {
       0)   0.691 us    |    __mutex_unlock_slowpath();
       0)   1.787 us    |  }
       0)               |  mutex_lock_interruptible_nested() {
       0)   0.548 us    |    __might_sleep();
       0)   1.945 us    |  }
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      f9349a8f
  16. 18 2月, 2009 3 次提交
    • F
      tracing/function-graph-tracer: trace the idle tasks · 5b058bcd
      Frederic Weisbecker 提交于
      When the function graph tracer is activated, it iterates over the task_list
      to allocate a stack to store the return addresses.
      
      But the per cpu idle tasks are not iterated by using
      do_each_thread / while_each_thread.
      
      So we have to iterate on them manually.
      
      This fixes somes weirdness in the traces and many losses of traces.
      Examples on two cpus:
      
       0)   Xorg-4287    |   2.906 us    |              }
       0)   Xorg-4287    |   3.965 us    |            }
       0)   Xorg-4287    |   5.302 us    |          }
       ------------------------------------------
       0)   Xorg-4287    =>    <idle>-0
       ------------------------------------------
      
       0)    <idle>-0    |   2.861 us    |                        }
       0)    <idle>-0    |   0.526 us    |                        set_normalized_timespec();
       0)    <idle>-0    |   7.201 us    |                      }
       0)    <idle>-0    |   8.214 us    |                    }
       0)    <idle>-0    |               |                    clockevents_program_event() {
       0)    <idle>-0    |               |                      lapic_next_event() {
       0)    <idle>-0    |   0.510 us    |                        native_apic_mem_write();
       0)    <idle>-0    |   1.546 us    |                      }
       0)    <idle>-0    |   2.583 us    |                    }
       0)    <idle>-0    | + 12.435 us   |                  }
       0)    <idle>-0    | + 13.470 us   |                }
       0)    <idle>-0    |   0.608 us    |                _spin_unlock_irqrestore();
       0)    <idle>-0    | + 23.270 us   |              }
       0)    <idle>-0    | + 24.336 us   |            }
       0)    <idle>-0    | + 25.417 us   |          }
       0)    <idle>-0    |   0.593 us    |          _spin_unlock();
       0)    <idle>-0    | + 41.869 us   |        }
       0)    <idle>-0    | + 42.906 us   |      }
       0)    <idle>-0    | + 95.035 us   |    }
       0)    <idle>-0    |   0.540 us    |    menu_reflect();
       0)    <idle>-0    | ! 100.404 us  |  }
       0)    <idle>-0    |   0.564 us    |  mce_idle_callback();
       0)    <idle>-0    |               |  enter_idle() {
       0)    <idle>-0    |   0.526 us    |    mce_idle_callback();
       0)    <idle>-0    |   1.757 us    |  }
       0)    <idle>-0    |               |  cpuidle_idle_call() {
       0)    <idle>-0    |               |    menu_select() {
       0)    <idle>-0    |   0.525 us    |      pm_qos_requirement();
       0)    <idle>-0    |   0.518 us    |      tick_nohz_get_sleep_length();
       0)    <idle>-0    |   2.621 us    |    }
      [...]
       1)    <idle>-0    |   0.518 us    |              touch_softlockup_watchdog();
       1)    <idle>-0    | + 14.355 us   |            }
       1)    <idle>-0    | + 22.840 us   |          }
       1)    <idle>-0    | + 25.949 us   |        }
       1)    <idle>-0    |               |        handle_irq() {
       1)    <idle>-0    |   0.511 us    |          irq_to_desc();
       1)    <idle>-0    |               |          handle_edge_irq() {
       1)    <idle>-0    |   0.638 us    |            _spin_lock();
       1)    <idle>-0    |               |            ack_apic_edge() {
       1)    <idle>-0    |   0.510 us    |              irq_to_desc();
       1)    <idle>-0    |               |              move_native_irq() {
       1)    <idle>-0    |   0.510 us    |                irq_to_desc();
       1)    <idle>-0    |   1.532 us    |              }
       1)    <idle>-0    |   0.511 us    |              native_apic_mem_write();
       ------------------------------------------
       1)    <idle>-0    =>    cat-5073
       ------------------------------------------
      
       1)    cat-5073    |   3.731 us    |                    }
       1)    cat-5073    |               |                    run_local_timers() {
       1)    cat-5073    |   0.533 us    |                      hrtimer_run_queues();
       1)    cat-5073    |               |                      raise_softirq() {
       1)    cat-5073    |               |                        __raise_softirq_irqoff() {
       1)    cat-5073    |               |                          /* nr: 1 */
       1)    cat-5073    |   2.718 us    |                        }
       1)    cat-5073    |   3.814 us    |                      }
      Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      5b058bcd
    • S
      ftrace: rename _hook to _probe · b6887d79
      Steven Rostedt 提交于
      Impact: clean up
      
      Ingo Molnar did not like the _hook naming convention used by the
      select function tracer. Luis Claudio R. Goncalves suggested using
      the "_probe" extension. This patch implements the change of
      calling the functions and variables "_hook" and replacing them
      with "_probe".
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      b6887d79
    • S
      ftrace: clean up coding style · 6a24a244
      Steven Rostedt 提交于
      Ingo Molnar pointed out some coding style issues with the recent ftrace
      updates. This patch cleans them up.
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      6a24a244
  17. 17 2月, 2009 5 次提交
    • I
      ftrace: fix !CONFIG_DYNAMIC_FTRACE ftrace_swapper_pid definition · 73d3fd96
      Ingo Molnar 提交于
      Impact: build fix
      Signed-off-by: NIngo Molnar <mingo@elte.hu>
      73d3fd96
    • S
      ftrace: add pretty print to selected fuction traces · 809dcf29
      Steven Rostedt 提交于
      This patch adds a call back for the tracers that have hooks to
      selected functions. This allows the tracer to show better output
      in the set_ftrace_filter file.
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      809dcf29
    • S
      ftrace: show selected functions in set_ftrace_filter · 8fc0c701
      Steven Rostedt 提交于
      This patch adds output to show what functions have tracer hooks
      attached to them.
      
        # echo 'sys_open:traceon:4' > /debug/tracing/set_ftrace_filter
        # cat set_ftrace_filter
      
       #### all functions enabled ####
       sys_open:ftrace_traceon:0000000000000004
      
        # echo 'do_fork:traceoff:' > set_ftrace_filter
        # cat set_ftrace_filter
      
       #### all functions enabled ####
       sys_open:ftrace_traceon:0000000000000002
       do_fork:ftrace_traceoff:ffffffffffffffff
      
      Note the 4 changed to a 2. This is because The code was executed twice
      since the traceoff was added. If a cat is done again:
      
       #### all functions enabled ####
       sys_open:ftrace_traceon
       do_fork:ftrace_traceoff:ffffffffffffffff
      
      The number disappears. That is because it will not print a NULL.
      
      Callbacks to allow the tracer to pretty print will be implemented soon.
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      8fc0c701
    • S
      ftrace: trace different functions with a different tracer · 59df055f
      Steven Rostedt 提交于
      Impact: new feature
      
      Currently, the function tracer only gives you an ability to hook
      a tracer to all functions being traced. The dynamic function trace
      allows you to pick and choose which of those functions will be
      traced, but all functions being traced will call all tracers that
      registered with the function tracer.
      
      This patch adds a new feature that allows a tracer to hook to specific
      functions, even when all functions are being traced. It allows for
      different functions to call different tracer hooks.
      
      The way this is accomplished is by a special function that will hook
      to the function tracer and will set up a hash table knowing which
      tracer hook to call with which function. This is the most general
      and easiest method to accomplish this. Later, an arch may choose
      to supply their own method in changing the mcount call of a function
      to call a different tracer. But that will be an exercise for the
      future.
      
      To register a function:
      
       struct ftrace_hook_ops {
      	void			(*func)(unsigned long ip,
      					unsigned long parent_ip,
      					void **data);
      	int			(*callback)(unsigned long ip, void **data);
      	void			(*free)(void **data);
       };
      
       int register_ftrace_function_hook(char *glob, struct ftrace_hook_ops *ops,
      				  void *data);
      
      glob is a simple glob to search for the functions to hook.
      ops is a pointer to the operations (listed below)
      data is the default data to be passed to the hook functions when traced
      
      ops:
       func is the hook function to call when the functions are traced
       callback is a callback function that is called when setting up the hash.
         That is, if the tracer needs to do something special for each
         function, that is being traced, and wants to give each function
         its own data. The address of the entry data is passed to this
         callback, so that the callback may wish to update the entry to
         whatever it would like.
       free is a callback for when the entry is freed. In case the tracer
         allocated any data, it is give the chance to free it.
      
      To unregister we have three functions:
      
        void
        unregister_ftrace_function_hook(char *glob, struct ftrace_hook_ops *ops,
      				void *data)
      
      This will unregister all hooks that match glob, point to ops, and
      have its data matching data. (note, if glob is NULL, blank or '*',
      all functions will be tested).
      
        void
        unregister_ftrace_function_hook_func(char *glob,
      				 struct ftrace_hook_ops *ops)
      
      This will unregister all functions matching glob that has an entry
      pointing to ops.
      
        void unregister_ftrace_function_hook_all(char *glob)
      
      This simply unregisters all funcs.
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      59df055f
    • S
      ftrace: consolidate mutexes · e6ea44e9
      Steven Rostedt 提交于
      Impact: clean up
      
      Now that ftrace_lock is a mutex, there is no reason to have three
      different mutexes protecting similar data. All the mutex paths
      are not in hot paths, so having a mutex to cover more data is
      not a problem.
      
      This patch removes the ftrace_sysctl_lock and ftrace_start_lock
      and uses the ftrace_lock to protect the locations that were protected
      by these locks. By doing so, this change also removes some of
      the lock nesting that was taking place.
      
      There are still more mutexes in ftrace.c that can probably be
      consolidated, but they can be dealt with later. We need to be careful
      about the way the locks are nested, and by consolidating, we can cause
      a recursive deadlock.
      Signed-off-by: NSteven Rostedt <srostedt@redhat.com>
      e6ea44e9