Currently, armpmu_enable iterates through the events for a given
counter set, calling armpmu->enable on each before calling
armpmu->start to start the PMU's counters.

As armpmu->enable is called when each event is added, each event is
already configured in hardware. Due to this, calling armpmu->enable
in armpmu_enable is unnecessary and confusing.

This patch removes the unnecessary calls to armpmu->enable.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently, struct arm_pmu and related functions are only visible to
{,arch/arm/}/kernel/perf_event.c. This prevents new drivers from using
the framework.

This patch  moves declarations to asm/pmu.h, allowing new PMU drivers
to use the framework.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently struct cpu_hw_events stores data on events running on a
PMU associated with a CPU. As this data is general enough to be used
for system PMUs, this name is a misnomer, and may cause confusion when
it is used for system PMUs.

Additionally, 'armpmu' is commonly used as a parameter name for an
instance of struct arm_pmu. The name is also used for a global instance
which represents the CPU's PMU.

As cpu_hw_events is now not tied to CPU PMUs, it is renamed to
pmu_hw_events, with instances of it renamed similarly. As the global
'armpmu' is CPU-specfic, it is renamed to cpu_pmu. This should make it
clearer which code is generic, and which is coupled with the CPU.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently the event accounting data in pmu_hw_events is stored in
fixed-sized arrays within the structure.

This patch refactors the accounting data to allow any number of events
to be managed.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently, a single static instance of struct pmu is used when
registering an ARM PMU with the main perf subsystem. This limits
the ARM perf code to supporting a single PMU.

This patch replaces the static struct pmu instance with a member
variable on struct arm_pmu. This provides bidirectional mapping
between the two structs, and therefore allows for support of multiple
PMUs. The function 'to_arm_pmu' is provided for convenience.

PMU-generic functions are also updated to use the new mapping, and
PMU-generic initialisation of the member variables is moved into a new
function: armpmu_init.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently mapping an event type to a hardware configuration value
depends on the data being pointed to from struct arm_pmu. These fields
(cache_map, event_map, raw_event_mask) are currently specific to CPU
PMUs, and do not serve the general case well.

This patch replaces the event map pointers on struct arm_pmu with a new
'map_event' function pointer. Small shim functions are used to reuse
the existing common code.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently, the ARM perf code assumes all PMUs it will handle are
CPU PMUs, having ARM_PMU_DEVICE_CPU hardcoded when reserving or
releasing hardware. This means that currently, the ARM perf code can't
support system PMUs.

This patch adds a 'type' field to struct arm_pmu, which allows the code
to reserve & release the hardware regardless of the PMU type.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently, a single lock serialises access to CPU PMU registers. This
global locking is unnecessary as PMU registers are local to the CPU
they monitor.

This patch replaces the global lock with a per-CPU lock. As the lock is
in struct cpu_hw_events, PMUs providing a single cpu_hw_events instance
can be locked globally.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

As armpmu_disable will call armpmu->stop when the last event has been
removed, this is pointless and simply adds to the noise when debugging.
Additionally, due to this call occurring in a preemptible context, this
is problematic for per-cpu locking of PMU registers (where we will
attempt to access per-cpu spinlock for use with raw_spin_lock_irqsave).

This patch removes the call to armpmu->stop.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently, cpu_hw_events is a global per-CPU variable. To enable
support for multiple PMUs, there needs to be a mapping from an instance
of arm_pmu to its cpu_hw_events. Additionally, as system PMUs are not
CPU-affine, they should not have this stored per-CPU.

This patch moves access to the hardware events data behind an accessor
function (arm_pmu::get_hw_events). This allows each instance to have
its own hardware event data, which can be stored per-CPU or globally as
required.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently the ARM perf code supports having a single struct
platform_device to supply IRQ numbers, limiting it to supporting a
single PMU.

This patch makes a platform_device instance variable on struct arm_pmu.
This should allow for multiple PMUs to be supported in future.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch moves the active_events counter into struct arm_pmu, in
preparation for supporting multiple PMUs. This also moves
pmu_reserve_mutex, as it is used to guard accesses to active_events.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently, pmu_hw_events::active_mask is used to keep track of which
events are active in hardware. As we can stop counters and their
interrupts, this is unnecessary.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently, event group validation compares each event's 'pmu' pointer
against the static 'pmu' pointer. This limits the code to supporting
only 1 PMU.

This patch changes the behaviour to consider an event's group leader's
'pmu' pointer as canonical for validation. This should ease later
generalisation of the code to support multiple PMUs at once.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently, an "empty" struct pmu is registered as the CPU PMU,
regardless of whether there is a physical PMU. This burdens the
accessor functions with checks to see whether a PMU is actually
present.

This patch changes initialisation to register a PMU only if there is a
supported PMU present, and removes the checks that this change makes
redundant.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Modern PMUs allow for mode exclusion, so we no longer wish to return
-EPERM if it is requested.

This patch provides a hook in the armpmu structure for implementing
mode exclusion. The hw_perf_event initialisation is slightly delayed so
that the backend code can update the structure if required.
Acked-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NJean Pihet <j-pihet@ti.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

ARM PMU code used to use 1-based indices for PMU registers. This caused
several data structures (pmu_hw_events::{active_events, used_mask, events})
to have an unused element at index zero. ARMPMU_MAX_HWEVENTS still takes
this indexing into account, and currently equates to 33.

This patch updates the core ARM perf code to use the 0th index again.
Acked-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NJean Pihet <j-pihet@ti.com>
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Commit 5dfc54e0 ("ARM: GIC: avoid routing interrupts to offline CPUs")
prevents the GIC from setting the affinity of an IRQ to a CPU with
id >= nr_cpu_ids. This was previously abused by perf on some platforms
where more IRQs were registered than possible CPUs.

This patch fixes the problem by using a cpumask_t to keep track of the
active (requested) interrupts in perf. The same effect could be achieved
by limiting the number of IRQs to the number of CPUs, but using a mask
instead will be useful for adding extended CPU hotplug support in the
future.
Acked-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NJean Pihet <j-pihet@ti.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Once upon a time, OProfile and Perf fought hard over who could play with
the PMU. To stop all hell from breaking loose, pmu.c offered an internal
reserve/release API and took care of parsing PMU platform data passed in
from board support code.

Now that Perf has ingested OProfile, let's move the platform device
handling into the Perf driver and out of the PMU locking code.
Unfortunately, the lock has to remain to prevent Perf being bitten by
out-of-tree modules such as LTTng, which still claim a right to the PMU
when Perf isn't looking.
Acked-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NJean Pihet <j-pihet@ti.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch removes const qualifiers from instances of struct arm_pmu,
and functions initialising them, in preparation for generalising
arm_pmu usage to system (AKA uncore) PMUs.

This will allow for dynamically modifiable structures (locks,
struct pmu) to be added as members of struct arm_pmu.
Acked-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NJean Pihet <j-pihet@ti.com>
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch adds support for the Cortex-A15 PMU to the ARMv7
perf-event backend.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch adds support for the Cortex-A5 PMU to the ARMv7 perf-event
backend.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

armpmu_enable can be called in situations where no events are present
(for example, from the event rotation tick after a profiled task has
exited). In this case, we currently start the PMU anyway which may
leave it active inevitably without any events being monitored.

This patch adds a simple check to the enabling code so that we avoid
starting the PMU when no events are present.

Cc: <stable@kernel.org>
Reported-by: NAshwin Chaugle <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Currently, PMU platform_device reservation relies on some minor abuse
of the platform_device::id field for determining the type of PMU. This
is problematic for device tree based probing, where the ID cannot be
controlled.

This patch removes reliance on the id field, and depends on each PMU's
platform driver to figure out which type it is. As all PMUs handled by
the current platform_driver name "arm-pmu" are CPU PMUs, this
convention is hardcoded. New PMU types can be supported through the use
of {of,platform}_device_id tables
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NJamie Iles <jamie@jamieiles.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Cc: Rob Herring <rob.herring@calxeda.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

When initialising a PMU, there is a check to protect against races with
other CPUs filling all of the available event slots. Since armpmu_add
checks that an event can be scheduled, we do not need to do this at
initialisation time. Furthermore the current code is broken because it
assumes that atomic_inc_not_zero will unconditionally increment
active_counts and then tries to decrement it again on failure.

This patch removes the broken, redundant code.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Cc: Jamie Iles <jamie@jamieiles.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The ARM user backtrace code can get into an infinite loop if it
runs into an invalid stack frame which points back to itself.
This situation has been observed in practice.  Fix it by capping
the number of entries in the backtrace.  This is also what other
architectures do in their backtrace code.
Signed-off-by: NSonny Rao <sonnyrao@chromium.org>
Acked-by: NJamie Iles <jamie@jamieiles.com>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Commit a737823d ("ARM: perf: ensure overflows aren't missed due to IRQ
latency") changed the way that event deltas are calculated on overflow
so that we don't miss events when the new count value overtakes the
previous one.

Unfortunately, we forget to count the event that passes through zero so
we end up being off by 1. This patch adds on the correction.
Reported-by: NChris Moore <moore@free.fr>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

If a counter overflows during a perf stat profiling run it may overtake
the last known value of the counter:

    0        prev     new                0xffffffff
    |----------|-------|----------------------|

In this case, the number of events that have occurred is
(0xffffffff - prev) + new. Unfortunately, the event update code will
not realise an overflow has occurred and will instead report the event
delta as (new - prev) which may be considerably smaller than the real
count.

This patch adds an extra argument to armpmu_event_update which indicates
whether or not an overflow has occurred. If an overflow has occurred
then we use the maximum period of the counter to calculate the elapsed
events.
Acked-by: NJamie Iles <jamie@jamieiles.com>
Reported-by: NAshwin Chaugule <ashwinc@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ARMv7 dictates that the interrupt-enable and count-enable registers for
each PMU counter are UNKNOWN following core reset.

This patch adds a new (optional) function pointer to struct arm_pmu for
resetting the PMU state during init. The reset function is called on
each CPU via an arch_initcall in the generic ARM perf_event code and
allows the PMU backend to write sane values to any UNKNOWN registers.
Acked-by: NJean Pihet <j-pihet@ti.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Allow a platform-specific IRQ handler to be specified via platform data.
This will be used to implement the single-irq workaround for the DB8500.
Signed-off-by: NRabin Vincent <rabin.vincent@stericsson.com>
Acked-by: NLee Jones <lee.jones@linaro.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

Since tail is the previous fp - 1, we need to compare the new fp with tail + 1
to ensure that we don't end up passing in the same tail again, in order to
avoid a potential infinite loop in the perf interrupt handler (which has been
observed to occur).  A similar fix seems to be needed in the OProfile code.
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRabin Vincent <rabin.vincent@stericsson.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Extend the perf_pmu_register() interface to allow for named and
dynamic pmu types.

Because we need to support the existing static types we cannot use
dynamic types for everything, hence provide a type argument.

If we want to enumerate the PMUs they need a name, provide one.
Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <20101117222056.259707703@chello.nl>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

For kernels built with PREEMPT_RT, critical sections protected
by standard spinlocks are preemptible. This is not acceptable
on perf as (a) we may be scheduled onto a different CPU whilst
reading/writing banked PMU registers and (b) the latency when
reading the PMU registers becomes unpredictable.

This patch upgrades the pmu_lock spinlock to a raw_spinlock
instead.
Reported-by: NJamie Iles <jamie@jamieiles.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Russell reported a number of warnings coming from sparse when
checking the ARM perf_event.c files:

| perf_event.c seems to also have problems too:
|
|   CHECK   arch/arm/kernel/perf_event.c
|   arch/arm/kernel/perf_event.c:37:1: warning: symbol 'pmu_lock' was not declared. Should it be static?
|   arch/arm/kernel/perf_event.c:70:1: warning: symbol 'cpu_hw_events' was not declared. Should it be static?
|   arch/arm/kernel/perf_event.c:1006:1: warning: symbol 'armv6pmu_enable_event' was not declared. Should it be static?
|   arch/arm/kernel/perf_event.c:1113:1: warning: symbol 'armv6pmu_stop' was not declared. Should it be static?
|   arch/arm/kernel/perf_event.c:1956:6: warning: symbol 'armv7pmu_enable_event' was not declared. Should it be static?
|   arch/arm/kernel/perf_event.c:3072:14: warning: incorrect type in argument 1 (different address spaces)
|   arch/arm/kernel/perf_event.c:3072:14:    expected void const volatile [noderef] <asn:1>*<noident>
|   arch/arm/kernel/perf_event.c:3072:14:    got struct frame_tail *tail
|   arch/arm/kernel/perf_event.c:3074:49: warning: incorrect type in argument 2 (different address spaces)
|   arch/arm/kernel/perf_event.c:3074:49:    expected void const [noderef] <asn:1>*from
|   arch/arm/kernel/perf_event.c:3074:49:    got struct frame_tail *tail

This patch resolves these issues so we can live in silence
again.
Reported-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The perf hardware pmu got initialized at various points in the boot,
some before early_initcall() some after (notably arch_initcall).

The problem is that the NMI lockup detector is ran from early_initcall()
and expects the hardware pmu to be present.

Sanitize this by moving all architecture hardware pmu implementations to
initialize at early_initcall() and move the lockup detector to an explicit
initcall right after that.

Cc: paulus <paulus@samba.org>
Cc: davem <davem@davemloft.net>
Cc: Michael Cree <mcree@orcon.net.nz>
Cc: Deng-Cheng Zhu <dengcheng.zhu@gmail.com>
Acked-by: NPaul Mundt <lethal@linux-sh.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <1290707759.2145.119.camel@laptop>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

The ARM perf_event.c file contains all PMU backends and, as new PMUs
are introduced, will continue to grow.

This patch follows the example of x86 and splits the PMU implementations
into separate files which are then #included back into the main
file. Compile-time guards are added to each PMU file to avoid compiling
in code that is not relevant for the version of the architecture which
we are targetting.
Acked-by: NJean Pihet <j-pihet@ti.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently, perf uses the PMU ID as an index into a string table
to look up the name of a given PMU.

This patch encodes the name of a PMU directly into the arm_pmu
structure so that PMU-specific code can be factored out into
separate files.
Acked-by: NJamie Iles <jamie@jamieiles.com>
Acked-by: NJean Pihet <j-pihet@ti.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

In preparation for separating the PMU-specific code, this patch adds
self-contained init functions to each PMU, therefore removing any
PMU-specific knowledge from the PMU-agnostic init_hw_perf_events
function.
Acked-by: NJamie Iles <jamie@jamieiles.com>
Acked-by: NJean Pihet <j-pihet@ti.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Unlike other pmu functions, armv6pmu_pmu_stop is not declared static.
This patch adds the missing keyword.
Acked-by: NJamie Iles <jamie.iles@jamieiles.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The functions for mapping PMU events (perf, cache and raw) are
common between all PMU types and differ only in the data on which
they operate.

This patch implements common definitions of these mapping functions
and changes the arm_pmu struct to hold pointers to the data which
they require. This is in anticipation of separating out the PMU-specific
code into separate files.
Acked-by: NJamie Iles <jamie.iles@jamieiles.com>
Acked-by: NJean Pihet <j-pihet@ti.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>