The synth_event trace code contains some almost identical functions
and some small functions that are called only once - consolidate the
common code into single functions and fold in the small functions to
simplify the code overall.

Link: http://lkml.kernel.org/r/d1c8d8ad124a653b7543afe801d38c199ca5c20e.1580506712.git.zanussi@kernel.orgSigned-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

The patch 'tracing: Fix histogram code when expression has same var as
value' added code to return an existing variable reference when
creating a new variable reference, which resulted in var_ref_vals
slots being reused instead of being duplicated.

The implementation of the trace action assumes that the end of the
var_ref_vals array starting at action_data.var_ref_idx corresponds to
the values that will be assigned to the trace params. The patch
mentioned above invalidates that assumption, which means that each
param needs to explicitly specify its index into var_ref_vals.

This fix changes action_data.var_ref_idx to an array of var ref
indexes to account for that.

Link: https://lore.kernel.org/r/1580335695.6220.8.camel@kernel.org

Fixes: 8bcebc77 ("tracing: Fix histogram code when expression has same var as value")
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Have trace_boot_add_synth_event() use the synth_event interface.

Also, rename synth_event_run_cmd() to synth_event_run_command() now
that trace_boot's version is gone.

Link: http://lkml.kernel.org/r/94f1fa0e31846d0bddca916b8663404b20559e34.1580323897.git.zanussi@kernel.orgAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Move all the tracing selftest configs to the bottom of the tracing menu.
There's no reason for them to be interspersed throughout.

Also, move the bootconfig menu to the top.
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Move the config that enables the mmiotracer with the other tracers such that
all the tracers are together.
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

The MMIO test module was by itself, move it to the other test modules. Also,
add the text "Test module" to PREEMPTIRQ_DELAY_TEST as that create a test
module as well.
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

The features that depend on the function tracer were spread out through the
tracing menu, pull them together as it is easier to manage.
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Add Documentation for creating and generating synthetic events from
modules.

Link: http://lkml.kernel.org/r/734bf8789ff8700000c9acde61a553427910ddb5.1580323897.git.zanussi@kernel.orgAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Add a test module that checks the basic functionality of the in-kernel
kprobe event command generation API by creating kprobe events from a
module.

Link: http://lkml.kernel.org/r/97e502b204f9dba948e3fa3a4315448298218787.1580323897.git.zanussi@kernel.orgAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Have trace_boot_add_kprobe_event() use the kprobe_event interface.

Also, rename kprobe_event_run_cmd() to kprobe_event_run_command() now
that trace_boot's version is gone.

Link: http://lkml.kernel.org/r/af5429d11291ab1e9a85a0ff944af3b2bcf193c7.1580323897.git.zanussi@kernel.orgAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Add functions used to generate kprobe event commands, built on top of
the dynevent_cmd interface.

kprobe_event_gen_cmd_start() is used to create a kprobe event command
using a variable arg list, and kretprobe_event_gen_cmd_start() does
the same for kretprobe event commands.  kprobe_event_add_fields() can
be used to add single fields one by one or as a group.  Once all
desired fields are added, kprobe_event_gen_cmd_end() or
kretprobe_event_gen_cmd_end() respectively are used to actually
execute the command and create the event.

Link: http://lkml.kernel.org/r/95cc4696502bb6017f9126f306a45ad19b4cc14f.1580323897.git.zanussi@kernel.orgAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Add a test module that checks the basic functionality of the in-kernel
synthetic event generation API by generating and tracing synthetic
events from a module.

Link: http://lkml.kernel.org/r/fcb4dd9eb9eefb70ab20538d3529d51642389664.1580323897.git.zanussi@kernel.orgAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Add an exported function named synth_event_trace(), allowing modules
or other kernel code to trace synthetic events.

Also added are several functions that allow the same functionality to
be broken out in a piecewise fashion, which are useful in situations
where tracing an event from a full array of values would be
cumbersome.  Those functions are synth_event_trace_start/end() and
synth_event_add_(next)_val().

Link: http://lkml.kernel.org/r/7a84de5f1854acf4144b57efe835ca645afa764f.1580323897.git.zanussi@kernel.orgAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Add functions used to generate synthetic event commands, built on top
of the dynevent_cmd interface.

synth_event_gen_cmd_start() is used to create a synthetic event
command using a variable arg list and
synth_event_gen_cmd_array_start() does the same thing but using an
array of field descriptors.  synth_event_add_field(),
synth_event_add_field_str() and synth_event_add_fields() can be used
to add single fields one by one or as a group.  Once all desired
fields are added, synth_event_gen_cmd_end() is used to actually
execute the command and create the event.

synth_event_create() does everything, including creating the event, in
a single call.

Link: http://lkml.kernel.org/r/38fef702fad5ef208009f459552f34a94befd860.1580323897.git.zanussi@kernel.orgAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Add an interface used to build up dynamic event creation commands,
such as synthetic and kprobe events.  Interfaces specific to those
particular types of events and others can be built on top of this
interface.

Command creation is started by first using the dynevent_cmd_init()
function to initialize the dynevent_cmd object.  Following that, args
are appended and optionally checked by the dynevent_arg_add() and
dynevent_arg_pair_add() functions, which use objects representing
arguments and pairs of arguments, initialized respectively by
dynevent_arg_init() and dynevent_arg_pair_init().  Finally, once all
args have been successfully added, the command is finalized and
actually created using dynevent_create().

The code here for actually printing into the dyn_event->cmd buffer
using snprintf() etc was adapted from v4 of Masami's 'tracing/boot:
Add synthetic event support' patch.

Link: http://lkml.kernel.org/r/1f65fa44390b6f238f6036777c3784ced1dcc6a0.1580323897.git.zanussi@kernel.orgSigned-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

create_or_delete_synth_event() contains code to delete a synthetic
event, which would be useful on its own - specifically, it would be
useful to allow event-creating modules to call it separately.

Separate out the delete code from that function and create an exported
function named synth_event_delete().

Link: http://lkml.kernel.org/r/050db3b06df7f0a4b8a2922da602d1d879c7c1c2.1580323897.git.zanussi@kernel.orgAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Add a function to get an event file and prevent it from going away on
module or instance removal.

trace_get_event_file() will find an event file in a given instance (if
instance is NULL, it assumes the top trace array) and return it,
pinning the instance's trace array as well as the event's module, if
applicable, so they won't go away while in use.

trace_put_event_file() does the matching release.

Link: http://lkml.kernel.org/r/bb31ac4bdda168d5ed3c4b5f5a4c8f633e8d9118.1580323897.git.zanussi@kernel.orgAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
[ Moved trace_array_put() to end of trace_put_event_file() ]
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Add a new trace_array_find() function that can be used to find a trace
array given the instance name, and replace existing code that does the
same thing with it.  Also add trace_array_find_get() which does the
same but returns the trace array after upping its refcount.

Also make both available for use outside of trace.c.

Link: http://lkml.kernel.org/r/cb68528c975eba95bee4561ac67dd1499423b2e5.1580323897.git.zanussi@kernel.orgAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

if seq_file .next fuction does not change position index,
read after some lseek can generate unexpected output.

Without patch:
 # dd bs=30 skip=1 if=/sys/kernel/tracing/events/sched/sched_switch/trigger
 dd: /sys/kernel/tracing/events/sched/sched_switch/trigger: cannot skip to specified offset
 n traceoff snapshot stacktrace enable_event disable_event enable_hist disable_hist hist
 # Available triggers:
 # traceon traceoff snapshot stacktrace enable_event disable_event enable_hist disable_hist hist
 6+1 records in
 6+1 records out
 206 bytes copied, 0.00027916 s, 738 kB/s

Notice the printing of "# Available triggers:..." after the line.

With the patch:
 # dd bs=30 skip=1 if=/sys/kernel/tracing/events/sched/sched_switch/trigger
 dd: /sys/kernel/tracing/events/sched/sched_switch/trigger: cannot skip to specified offset
 n traceoff snapshot stacktrace enable_event disable_event enable_hist disable_hist hist
 2+1 records in
 2+1 records out
 88 bytes copied, 0.000526867 s, 167 kB/s

It only prints the end of the file, and does not restart.

Link: http://lkml.kernel.org/r/3c35ee24-dd3a-8119-9c19-552ed253388a@virtuozzo.com

https://bugzilla.kernel.org/show_bug.cgi?id=206283Signed-off-by: NVasily Averin <vvs@virtuozzo.com>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

if seq_file .next fuction does not change position index,
read after some lseek can generate unexpected output.

Link: http://lkml.kernel.org/r/7ad85b22-1866-977c-db17-88ac438bc764@virtuozzo.comSigned-off-by: NVasily Averin <vvs@virtuozzo.com>
[ This is not a bug fix, it just makes it "technically correct"
  which is why I applied it. NULL is only returned on an anomaly
  which triggers a WARN_ON ]
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

if seq_file .next fuction does not change position index,
read after some lseek can generate unexpected output.

Without patch:
 # dd bs=4 skip=1 if=/sys/kernel/tracing/set_ftrace_pid
 dd: /sys/kernel/tracing/set_ftrace_pid: cannot skip to specified offset
 id
 no pid
 2+1 records in
 2+1 records out
 10 bytes copied, 0.000213285 s, 46.9 kB/s

Notice the "id" followed by "no pid".

With the patch:
 # dd bs=4 skip=1 if=/sys/kernel/tracing/set_ftrace_pid
 dd: /sys/kernel/tracing/set_ftrace_pid: cannot skip to specified offset
 id
 0+1 records in
 0+1 records out
 3 bytes copied, 0.000202112 s, 14.8 kB/s

Notice that it only prints "id" and not the "no pid" afterward.

Link: http://lkml.kernel.org/r/4f87c6ad-f114-30bb-8506-c32274ce2992@virtuozzo.com

https://bugzilla.kernel.org/show_bug.cgi?id=206283Signed-off-by: NVasily Averin <vvs@virtuozzo.com>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Reading the sched_cmdline_ref and sched_tgid_ref initial state within
tracing_start_sched_switch without holding the sched_register_mutex is
racy against concurrent updates, which can lead to tracepoint probes
being registered more than once (and thus trigger warnings within
tracepoint.c).

[ May be the fix for this bug ]
Link: https://lore.kernel.org/r/000000000000ab6f84056c786b93@google.com

Link: http://lkml.kernel.org/r/20190817141208.15226-1-mathieu.desnoyers@efficios.com

Cc: stable@vger.kernel.org
CC: Steven Rostedt (VMware) <rostedt@goodmis.org>
CC: Joel Fernandes (Google) <joel@joelfernandes.org>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Thomas Gleixner <tglx@linutronix.de>
CC: Paul E. McKenney <paulmck@linux.ibm.com>
Reported-by: syzbot+774fddf07b7ab29a1e55@syzkaller.appspotmail.com
Fixes: d914ba37 ("tracing: Add support for recording tgid of tasks")
Signed-off-by: NMathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Move external function declarations into kernel/trace/trace.h
from trace_boot.c for tracing subsystem internal use.

Link: http://lkml.kernel.org/r/158029060405.12381.11944554430359702545.stgit@devnote2Signed-off-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Include some required (but currently indirectly included)
headers and sort it alphabetically.

Link: http://lkml.kernel.org/r/158029059514.12381.6597832266860248781.stgit@devnote2Signed-off-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Add a testcase ensuring that the tracing error_log correctly displays
hist trigger parsing errors.

Link: http://lkml.kernel.org/r/62ec58d9aca661cde46ba678e32a938427945e9e.1561743018.git.zanussi@kernel.orgReviewed-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

The 'hist:' prefix gets stripped from the command text during command
processing, but should be added back when displaying the command
during error processing.

Not only because it's what should be displayed but also because not
having it means the test cases fail because the caret is miscalculated
by the length of the prefix string.

Link: http://lkml.kernel.org/r/449df721f560042e22382f67574bcc5b4d830d3d.1561743018.git.zanussi@kernel.orgReviewed-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Add error codes and messages for all the error paths leading to sort
specification parsing errors.

Link: http://lkml.kernel.org/r/237830dc05e583fbb53664d817a784297bf961be.1561743018.git.zanussi@kernel.orgReviewed-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

In the process of adding better error messages for sorting, I realized
that strsep was being used incorrectly and some of the error paths I
was expecting to be hit weren't and just fell through to the common
invalid key error case.

It also became obvious that for keyword assignments, it wasn't
necessary to save the full assignment and reparse it later, and having
a common empty-assignment check would also make more sense in terms of
error processing.

Change the code to fix these problems and simplify it for new error
message changes in a subsequent patch.

Link: http://lkml.kernel.org/r/1c3ef0b6655deaf345f6faee2584a0298ac2d743.1561743018.git.zanussi@kernel.org

Fixes: e62347d2 ("tracing: Add hist trigger support for user-defined sorting ('sort=' param)")
Fixes: 7ef224d1 ("tracing: Add 'hist' event trigger command")
Fixes: a4072fe8 ("tracing: Add a clock attribute for hist triggers")
Reported-by: NMasami Hiramatsu <mhiramat@kernel.org>
Reviewed-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NTom Zanussi <zanussi@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Sphinx warnings about his:

	Documentation/trace/kprobetrace.rst:68: WARNING: undefined label: user_mem_access (if the link has no caption the label must precede a section header)

The problem is quite simple: Sphinx wants a blank line after
references.

Link: http://lkml.kernel.org/r/a83ea390bc28784518fce772b4c961ea1c976f14.1560883872.git.mchehab+samsung@kernel.orgAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NMauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

As warnings can trigger panics, especially when "panic_on_warn" is set,
memory failure warnings can cause panics and fail fuzz testers that are
stressing memory.

Create a MEM_FAIL() macro to use instead of WARN() in the tracing code
(perhaps this should be a kernel wide macro?), and use that for memory
failure issues. This should stop failing fuzz tests due to warnings.

Link: https://lore.kernel.org/r/CACT4Y+ZP-7np20GVRu3p+eZys9GPtbu+JpfV+HtsufAzvTgJrg@mail.gmail.comSuggested-by: NDmitry Vyukov <dvyukov@google.com>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

The trace_array_get_by_name() creates a ftrace instance and
trace_array_put() is used to remove the reference. Even though the
trace_array_get_by_name() creates the instance, it also adds a reference
count to it, that prevents user space from removing it.

As the bootconfig just creates the instance on boot up, it should still be
used where it can be deleted by user space after boot. A trace_array_put()
is required to let that happen.

Also, change the documentation on trace_array_get_by_name() to make this not
be so confusing.

Link: https://lore.kernel.org/r/20200124205927.76128804@rorschach.local.home

Fixes: 4f712a4d ("tracing/boot: Add instance node support")
Acked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

commit f42c85e7 moved tracepoint's ftrace
creation into include/trace/ftrace.h and trace/define_trace.h was deleted
as a result. However some comment info does not adapt to the change, which
is such a misguiding when reading related code.

This patch fix this by moving trace/trace_events.h to <trace/events/XXX.h>,
since tracepoint headers have already been moved to tarce/events/.

Link: http://lkml.kernel.org/r/1425419298-61941-1-git-send-email-houpengyang@huawei.comSigned-off-by: NHou Pengyang <houpengyang@huawei.com>
[ Pulled from the archeological digging of my INBOX ]
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

We checked "iter->trace" earlier so there is no need to check here.

Link: http://lkml.kernel.org/r/20141122183012.GB6994@mwandaSigned-off-by: NDan Carpenter <dan.carpenter@oracle.com>
[ Pulled from the archeological digging of my INBOX ]
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

I noticed when trying to use the trace-cmd python interface that reading the raw
buffer wasn't working for kernel_stack events.  This is because it uses a
stubbed version of __dynamic_array that doesn't do the __data_loc trick and
encode the length of the array into the field.  Instead it just shows up as a
size of 0.  So change this to __array and set the len to FTRACE_STACK_ENTRIES
since this is what we actually do in practice and matches how user_stack_trace
works.

Link: http://lkml.kernel.org/r/1411589652-1318-1-git-send-email-jbacik@fb.comSigned-off-by: NJosef Bacik <jbacik@fb.com>
[ Pulled from the archeological digging of my INBOX ]
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

tracing_stat_init() was always returning '0', even on the error paths.  It
now returns -ENODEV if tracing_init_dentry() fails or -ENOMEM if it fails
to created the 'trace_stat' debugfs directory.

Link: http://lkml.kernel.org/r/1410299381-20108-1-git-send-email-luis.henriques@canonical.com

Fixes: ed6f1c99 ("tracing: Check return value of tracing_init_dentry()")
Signed-off-by: NLuis Henriques <luis.henriques@canonical.com>
[ Pulled from the archeological digging of my INBOX ]
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

Looking through old emails in my INBOX, I came across a patch from Luis
Henriques that attempted to fix a race of two stat tracers registering the
same stat trace (extremely unlikely, as this is done in the kernel, and
probably doesn't even exist). The submitted patch wasn't quite right as it
needed to deal with clean up a bit better (if two stat tracers were the
same, it would have the same files).

But to make the code cleaner, all we needed to do is to keep the
all_stat_sessions_mutex held for most of the registering function.

Link: http://lkml.kernel.org/r/1410299375-20068-1-git-send-email-luis.henriques@canonical.com

Fixes: 002bb86d ("tracing/ftrace: separate events tracing and stats tracing engine")
Reported-by: NLuis Henriques <luis.henriques@canonical.com>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

This limitation are never lunched from introduce commit 970988e1
("tracing/kprobe: Add kprobe_event= boot parameter")

Could we remove it if no intention to implement it?

Link: http://lkml.kernel.org/r/1579586075-45132-1-git-send-email-alex.shi@linux.alibaba.comAcked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NAlex Shi <alex.shi@linux.alibaba.com>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

If we exit due to a bad input to trace_printk() (highly unlikely), then the
buffer variable will not be initialized when we unnest the ring buffer.
Reported-by: Nkbuild test robot <lkp@intel.com>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

The trace_array_get_by_name() function doesn't return error pointers,
it returns NULL on error.

Link: http://lkml.kernel.org/r/20200117053007.5h2juv272pokqhtq@kili.mountain

Fixes: 4f712a4d ("tracing/boot: Add instance node support")
Acked-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

This macro isn't used from commit 3a161d99 ("tracing: Create
seq_buf layer in trace_seq"). so no needs to keep it.

Link: http://lkml.kernel.org/r/1579586086-45543-1-git-send-email-alex.shi@linux.alibaba.comSigned-off-by: NAlex Shi <alex.shi@linux.alibaba.com>
Signed-off-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>