The 'perf ftrace' command fails to reset tracer after finishing
recording like below:

  $ sudo perf ftrace -v hello
  write 'nop' to tracing/current_tracer failed: Device or resource busy
  ...

This is because the trace_pipe file is open in pager process.  Move the
pager setup to before opening the file.
Signed-off-by: NNamhyung Kim <namhyung@kernel.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Fixes: 58335964 ("perf ftrace: Use pager for displaying result")
Link: http://lkml.kernel.org/r/20170618142302.25390-2-namhyung@kernel.orgSigned-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>

It'd be better for debugging to show an error message when it fails to
setup ftrace for some reason.
Signed-off-by: NNamhyung Kim <namhyung@kernel.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170618142302.25390-1-namhyung@kernel.orgSigned-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>

Not needed in this header, added to the places that need poll(), wait()
and a few other prototypes.

Link: http://lkml.kernel.org/n/tip-i39c7b6xmo1vwd9wxp6fmkl0@git.kernel.orgSigned-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>

Instead of getting it out of luck from util.h, where it isn't needed at
all.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-0bqugg5lc5ksla1v4m0dnmc1@git.kernel.orgSigned-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>

Removing it from util.h, part of an effort to disentangle the includes
hell, that makes changes to util.h or something included by it to cause
a complete rebuild of the tools.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-ztrjy52q1rqcchuy3rubfgt2@git.kernel.orgSigned-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>

We got it from the git sources but never used it for anything, with the
place where this would be somehow used remaining:

  static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
  {
	prefix = NULL;
	if (p->option & RUN_SETUP)
		prefix = NULL; /* setup_perf_directory(); */

Ditch it.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-uw5swz05vol0qpr32c5lpvus@git.kernel.orgSigned-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>

It's convenient to use the pager when seeing many lines of result.

Note that setup_pager() should be called after perf_evlist__prepare_workload()
since they can interfere each other regarding shared stdio streams.
Signed-off-by: NNamhyung Kim <namhyung@kernel.org>
Tested-by: NArnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170224011251.14946-4-namhyung@kernel.orgSigned-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>

The -a/--all-cpus and -C/--cpu option is for controlling tracing cpus.
Signed-off-by: NNamhyung Kim <namhyung@kernel.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170224011251.14946-3-namhyung@kernel.orgSigned-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>

The -p (--pid) option enables to trace existing process by its pid.

Committer notes:

Testing it:

Using the function_graph tracer on a process that is just waiting for user
input and thus will make 'perf ftrace' sit there waiting for that, then press
any key on that mutt session and see what happens:

  # perf ftrace -t function_graph -p `pidof mutt` | head -40
  2)   1.038 us    |  switch_mm_irqs_off();
  ------------------------------------------
  2)    <idle>-0    =>   mutt-3595
  ------------------------------------------

  2)               |              finish_task_switch() {
  2)               |                smp_irq_work_interrupt() {
  2)               |                  irq_enter() {
  2)   0.180 us    |                    rcu_irq_enter();
  2)   1.248 us    |                  }
  2)               |                  __wake_up() {
  2)   0.126 us    |                    _raw_spin_lock_irqsave();
  2)               |                    __wake_up_common() {
  2)               |                      pollwake() {
  2)               |                        default_wake_function() {
  2)               |                          try_to_wake_up() {
  2)   0.662 us    |                            _raw_spin_lock_irqsave();
  2)               |                            select_task_rq_fair() {
  2)   1.719 us    |                              effective_load.isra.41();
  2)   1.343 us    |                              effective_load.isra.41();
  2)               |                              select_idle_sibling() {
  2)   0.331 us    |                                idle_cpu();
  2)   1.458 us    |                              }
  2)   8.350 us    |                            }
  2)   0.200 us    |                            _raw_spin_lock();
  2)               |                            ttwu_do_activate() {
  2)               |                              activate_task() {
  2)   0.136 us    |                                update_rq_clock.part.77();
  2)               |                                enqueue_task_fair() {
  2)               |                                  enqueue_entity() {
  2)   0.146 us    |                                    update_curr();
  2)   0.330 us    |                                    account_entity_enqueue();
  2)   0.280 us    |                                    update_cfs_shares();
  2)   0.321 us    |                                    place_entity();
  2)   0.206 us    |                                    __enqueue_entity();
  2)   6.926 us    |                                  }
  2)               |                                  enqueue_entity() {
  2)   0.105 us    |                                    update_curr();
  2)   0.175 us    |                                    account_entity_enqueue();
  2)   0.531 us    |                                    update_cfs_shares();
 #
Signed-off-by: NNamhyung Kim <namhyung@kernel.org>
Tested-by: NArnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170224011251.14946-1-namhyung@kernel.orgSigned-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>

Currently 'perf ftrace' command allows selecting 'function_graph' or
'function', defaulting to 'function_graph'.

Add ftrace.tracer config option to select the default tracer:

    # cat ~/.perfconfig
    [ftrace]
        tracer = function

    # perf ftrace usleep 123456 | head -10
      <...>-14450 [002] d... 10089.284231: finish_task_switch <-__schedule
      <...>-14450 [002] .... 10089.284232: finish_wait <-pipe_wait
      <...>-14450 [002] .... 10089.284232: mutex_lock <-pipe_wait
      <...>-14450 [002] .... 10089.284232: _cond_resched <-mutex_lock

Committer notes:

Retesting it with invalid variables, invalid values for ftrace.tracer,
and a valid one:

  # cat ~/.perfconfig
  [ftrace]
        trace = function
  # perf ftrace usleep 1
  Error: wrong config key-value pair ftrace.trace=function
  # cat ~/.perfconfig
  [ftrace]
        tracer = functin
  # perf ftrace usleep 1
  Please select "function_graph" (default) or "function"
  Error: wrong config key-value pair ftrace.tracer=functin
  # cat ~/.perfconfig
  [ftrace]
        tracer = function
  # perf ftrace usleep 1 | head -5
          <idle>-0     [000] d...  3855.820847: switch_mm_irqs_off <-__schedule
           <...>-18550 [000] d...  3855.820849: finish_task_switch <-__schedule
           <...>-18550 [000] d...  3855.820851: smp_irq_work_interrupt <-irq_work_interrupt
           <...>-18550 [000] d...  3855.820851: irq_enter <-smp_irq_work_interrupt
           <...>-18550 [000] d...  3855.820851: rcu_irq_enter <-irq_enter
  #
Signed-off-by: NTaeung Song <treeze.taeung@gmail.com>
Tested-by: NArnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1485862711-20216-3-git-send-email-treeze.taeung@gmail.com
[ Added missign space in error message, changed the logic to make it more compact and less error prone ]
Signed-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>

As a result of commit a3497642c261 ("perf ftrace: Make 'function_graph'
be the default tracer") the ftrace.tracer variable can't be NULL but the
other code setting default tracer remained.
Signed-off-by: NTaeung Song <treeze.taeung@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1485423339-22780-1-git-send-email-treeze.taeung@gmail.comSigned-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>

So that we can suppress the '-t function_graph' and get a more compact command
line:

  # perf ftrace usleep 123456 | grep raw_spin_lock | sort -k2 -nr | head -5
  2)   0.555 us    |                _raw_spin_lock();
  2)   0.516 us    |          _raw_spin_lock();
  2)   0.410 us    |          _raw_spin_lock_irq();
  2)   0.374 us    |                        _raw_spin_lock_irqsave();
  #
Tested-by: NMasami Hiramatsu <mhiramat@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jeremy Eder <jeder@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-ss9xgx5htpxcv86x42pnh3m6@git.kernel.orgSigned-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>

The 'perf ftrace' command is a simple wrapper of kernel's ftrace
functionality.  It only supports single thread tracing currently and
just reads trace_pipe in text and then write it to stdout.

Committer notes:

Testing it:

  # perf ftrace -f function_graph usleep 123456
  <SNIP>
  2)               |  SyS_nanosleep() {
  2)               |    _copy_from_user() {
  <SNIP>
  2)   0.900 us    |      }
  2)   1.354 us    |    }
  2)               |    hrtimer_nanosleep() {
  2)   0.062 us    |      __hrtimer_init();
  2)               |      do_nanosleep() {
  2)               |        hrtimer_start_range_ns() {
  <SNIP>
  2)   5.025 us    |        }
  2)               |        schedule() {
  2)   0.125 us    |          rcu_note_context_switch();
  2)   0.057 us    |          _raw_spin_lock();
  2)               |          deactivate_task() {
  2)   0.369 us    |            update_rq_clock.part.77();
  2)               |            dequeue_task_fair() {
  <SNIP>
  2) + 22.453 us   |            }
  2) + 23.736 us   |          }
  2)               |          pick_next_task_fair() {
  <SNIP>
  2) + 47.167 us   |          }
  2)               |          pick_next_task_idle() {
  <SNIP>
  2)   4.462 us    |          }
  ------------------------------------------
  2)  usleep-20387  =>    <idle>-0
  ------------------------------------------

  2)   0.806 us    |  switch_mm_irqs_off();
  ------------------------------------------
  2)    <idle>-0    =>  usleep-20387
  ------------------------------------------

  2)   0.151 us    |          finish_task_switch();
  2) @ 123597.2 us |        }
  2)   0.037 us    |        _cond_resched();
  2)               |        hrtimer_try_to_cancel() {
  2)   0.064 us    |          hrtimer_active();
  2)   0.353 us    |        }
  2) @ 123605.3 us |      }
  2) @ 123606.2 us |    }
  2) @ 123608.3 us |  } /* SyS_nanosleep */
  2)               |  __do_page_fault() {
 <SNIP>
Signed-off-by: NNamhyung Kim <namhyung@kernel.org>
Tested-by: NArnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: NMasami Hiramatsu <mhiramat@kernel.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jeremy Eder <jeder@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>,
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Link: http://lkml.kernel.org/n/tip-r1hgmsj4dxny8arn3o9mw512@git.kernel.org
[ Various foward port fixes, add man page ]
Signed-off-by: NArnaldo Carvalho de Melo <acme@redhat.com>