This is the ARM part of Christoph's patchset cleaning up the various
uses of __get_cpu_var across the tree.

The idea is to convert __get_cpu_var into either an explicit address
calculation using this_cpu_ptr() or into a use of this_cpu operations
that use the offset. Thereby address calculations are avoided and fewer
registers are used when code is generated.

[will: fixed debug ref counting checks and pcpu array accesses]
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NChristoph Lameter <cl@linux.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications.  For example, the fix in
commit 5e427ec2 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.

After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out.  Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.

Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
and are flagged as __cpuinit  -- so if we remove the __cpuinit from
the arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
related content into no-ops as early as possible, since that will get
rid of these warnings.  In any case, they are temporary and harmless.

This removes all the ARM uses of the __cpuinit macros from C code,
and all __CPUINIT from assembly code.  It also had two ".previous"
section statements that were paired off against __CPUINIT
(aka .section ".cpuinit.text") that also get removed here.

[1] https://lkml.org/lkml/2013/5/20/589

Cc: Russell King <linux@arm.linux.org.uk>
Cc: Will Deacon <will.deacon@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>

We must not declare dbg_cpu_pm_nb as __cpuinitdata as we need it after
system initialization for Suspend and CPUIdle.

This was done in commit 9a6eb310 ("ARM: hw_breakpoint: Debug powerdown
support for self-hosted debug").

Cc: stable@vger.kernel.org
Cc: Dietmar Eggemann <Dietmar.Eggemann@arm.com>
Signed-off-by: NBastian Hecht <hechtb+renesas@gmail.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

CPU debug features like hardware break, watchpoints can be used only
when the debug mode is enabled and available. Unfortunately on OMAP4
based devices, after a CPU power cycle, the debug feature gets disabled
which leads to a flood of messages coming from reset_ctrl_regs() which
gets called on every CPU_PM_EXIT with CPUidle enabled.

So make use of warn_once() so that system is usable.

Thanks to Will for pointers and Lokesh for the analysis of the issue.
Tested-by: NLokesh Vutla <lokeshvutla@ti.com>
Signed-off-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We must mask out the CPU_TASKS_FROZEN bit so that reset_ctrl_regs is
also called on a secondary CPU during s2ram resume, where only the boot
CPU will receive the PM_EXIT notification.
Signed-off-by: NDietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch introduces debug powerdown support for self-hosted debug for v7
and v7.1 debug architecture for a SinglePower system, i.e. a system without a
separate core and debug power domain. On a SinglePower system the OS Lock is
lost over a powerdown.

If CONFIG_CPU_PM is set the new function pm_init() registers hw_breakpoint
with CPU PM for a system supporting OS Save and Restore.

Receiving a CPU PM EXIT notifier indicates that a single CPU has exited a low
power state. A call to reset_ctrl_regs() is hooked into the CPU PM EXIT
notifier chain. This function makes sure that the sticky power-down is clear
(only v7 debug), the OS Double Lock is clear (only v7.1 debug) and it clears
the OS Lock for v7 debug (for a system supporting OS Save and Restore) and
v7.1 debug. Furthermore, it clears any vector-catch events and all
breakpoint/watchpoint control/value registers for v7 and v7.1 debug.
Signed-off-by: NDietmar Eggemann <dietmar.eggemann@arm.com>
[will: removed redundant has_ossr check]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

v7 debug introduced OS Save and Restore mechanism. On a v7 debug SinglePower
system, i.e a system without a separate core and debug power domain, which does
not support external debug over powerdown, it is implementation defined whether
OS Save and Restore is implemented.
v7.1 debug requires OS Save and Restore mechanism. v6 debug and v6.1 debug do
not implement it.

A new global variable bool has_ossr is introduced and is determined in
arch_hw_breakpoint_init() like debug_arch or the number of BRPs/WRPs.

The logic how to check if OS Save and Restore is supported has changed with
this patch. In reset_ctrl_regs() a mask consisting of OSLM[1] (OSLSR.3) and
OSLM[0] (OSLSR.0) was used to check if the system supports OS Save and
Restore. In the new function core_has_os_save_restore() only OSLM[0] is used.
It is not necessary to check OSLM[1] too since it is v7.1 debug specific and
v7.1 debug requires OS Save and Restore and thus OS Lock.
Signed-off-by: NDietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Coresight components and debug are using a common lock control mechansim.
Writing 0xC5ACCE55 to the Lock Access Register (LAR) in case of a coresight
components enables further access to the coresight device registers. Writing
any other value to it removes the write access.
Writing 0xC5ACCE55 to the OS Lock Access Register (OSLAR) in case of debug
locks the debug register for further access to the debug registers. Writing
any other value to it unlocks the debug registers.

Unfortunately, the existing coresight code uses the terms lock and unlock the
other way around. Unlocking stands for enabling write access and locking for
removing write access.

That is why the definition of the LAR and OSLAR key value has been changed to
CS_LAR_KEY.
Signed-off-by: NDietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

WARN_ONCE is a bit OTT for some of the simple failure cases encountered
in hw_breakpoint, so use either pr_warning or pr_warn_once instead.
Reported-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The coprocessor register CRn for accesses to the debug register can be a
different one than C0. Take this into account for the ARM_DBG_READ and
the ARM_DBG_WRITE macro.

The inline assembler calls which used a coprocessor register CRn other
than C0 are replaced by the ARM_DBG_READ or ARM_DBG_WRITE macro.
Tested-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NDietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Rather than attempt to enable monitor mode explicitly when scheduling in
a breakpoint event (which could raise an undefined exception trap when
accessing DBGDSCRext), instead check that DBGDSCRint.MDBGen is set
during event validation and report an error to the caller if not.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Booting on a v6 core without the CPUID feature registers (e.g. 1136)
leads to a noisy dmesg complaining about their absence.

This patch changes the pr_warning into a pr_warn_once to keep the log
quieter.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

v6 cores do not provide a way to clear the debug registers without first
enabling monitor mode, meaning that we could take spurious debug
exceptions. Instead, rely on the registers being in a sane state when we
boot as they are defined to be disabled out of reset anyway.
Tested-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The debug register reset sequence for v7 and v7.1 is congruent with
tap-dancing through a minefield.

Rather than wait until we've blown ourselves to pieces, this patch
instead checks the debug_err_mask after each potentially faulting
operation. We also move the enabling of monitor_mode to the end of the
sequence in order to prevent spurious debug events generated by UNKNOWN
register values.
Reported-by: NStephen Boyd <sboyd@codeaurora.org>
Tested-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Detecting whether halting debug is enabled is no longer possible via
the DBGDSCR in v7.1, returning an UNKNOWN value for the HDBGen bit via
CP14 when the OS lock is clear.

This patch removes the halting mode check and ensures that accesses to
the internal and external views of the DBGDSCR are serialised with an
instruction barrier.
Tested-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The OS save and restore register are optional in debug architecture v7,
so check the status register before attempting to clear the OS lock.
Tested-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Breakpoint validation currently fails for single-byte watchpoints on
addresses ending in 11b. There is no reason to forbid such a watchpoint,
so extend the validation code to allow it.

Cc: Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

From ARM debug architecture v7.1 onwards, a watchpoint exception causes
the DFAR to be updated with the faulting data address. However, DFSR.WnR
takes an UNKNOWN value and therefore cannot be used in general to
determine the access type that triggered the watchpoint.

This patch forbids watchpoints without an overflow handler from
specifying a specific access type (load/store). Those with overflow
handlers must be able to handle false positives potentially triggered by
a watchpoint of a different access type on the same address. For
SIGTRAP-based handlers (i.e. ptrace), this should have no impact.

Cc: <stable@vger.kernel.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Disintegrate asm/system.h for ARM.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
cc: Russell King <linux@arm.linux.org.uk>
cc: linux-arm-kernel@lists.infradead.org

The DFSR and IFSR register format is different when LPAE is enabled. In
addition, DFSR and IFSR have similar definitions for the fault type.
This modifies the fault code to correctly handle the new format.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

ARMv6 cores do not implement the DBGOSLAR register, so we don't need to
try and clear it on boot. Furthermore, the VCR is zeroed out of reset,
so we don't need to zero it explicitly when a CPU comes online.
Tested-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The ARM hw_breakpoint backend is currently a bit too noisy when things
start to go awry.

This patch removes a couple of over-zealous WARN_ONCE invocations and
replaces then with pr_warnings instead.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The ARM debug registers can only be accessed if the DBGSWENABLE signal
to the core is driven HIGH by the DAP. The architecture does not provide
a way to detect the value of this signal, so the best we can do is
register an undef_hook to trap debug register co-processor accesses and
then fail if the trap is taken.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

ARM debug architecture 7.1 mandates that the DFAR is updated on a
watchpoint debug exception to contain the faulting virtual address
of the memory access. This allows us to determine which watchpoints
have fired and therefore report useful information to userspace.

This patch adds support for using the DFAR in the watchpoint handler,
which allows us to support multiple watchpoints on CPUs implementing
the new debug architecture.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The current hw_breakpoint code on ARM reserves 1 breakpoint for each
watchpoint that is available. Since debug architectures prior to 7.1
are restricted to 1 watchpoint anyway, only one breakpoint was ever
reserved.

This patch changes the reservation strategy so that a single breakpoint
is reserved, regardless of the number of watchpoints. This is in
preparation for multiple-watchpoint support on debug architectures
from 7.1 onwards.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch adds initial support for Cortex-A15 (debug architecture v7.1)
to the hw_breakpoint ARM backend.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Avoid enabling interrupts if the parent context had interrupts enabled
in the abort handler assembly code, and move this into the breakpoint/
page/alignment fault handlers instead.

This gets rid of some special-casing for the breakpoint fault handlers
from the low level abort handler path.
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The DBGVCR, used for configuring vector catch debug events, is UNKNOWN
out of reset on ARMv7. When enabling monitor mode, this must be zeroed
to avoid UNPREDICTABLE behaviour.

This patch adds the zeroing code to the debug reset path.

Cc: stable <stable@kernel.org>
Reported-by: NStepan Moskovchenko <stepanm@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

These warnings are missing newlines and spaces causing confusing
looking output when they trigger.
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ARMv7 allows the debug core logic to be powered down and provides the
DBGPRSR register so that software can power-up and check the status of
the logic.

This patch ensures that the debug logic is powered up on ARMv7 cores
before we attempt to access the extended debug registers.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The ptrace debug information register was advertising breakpoint and
watchpoint resources for unsupported debug architectures. This meant
that setting breakpoints on these architectures would appear to succeed,
although they would never fire in reality.

This patch fixes the breakpoint slot probing so that it returns 0 when
running on an unsupported debug architecture.
Reported-by: NUlrich Weigand <ulrich.weigand@de.ibm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Reading baseline CP14 registers, other than DBGDIDR, when the OS Lock
is set leads to UNPREDICTABLE behaviour.

This patch ensures that we clear the OS lock before accessing anything
other than the DBGDIDR, thereby avoiding this behaviour.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The debug registers can only be manipulated from software if monitor
debug mode is enabled. On some cores, this can never be enabled (i.e.
the corresponding bit in the DSCR is RAZ/WI).

This patch ensures we can handle this hardware configuration and fail
gracefully, rather than blow up the kernel during boot.
Reported-by: NCyril Chemparathy <cyril@ti.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

sparse doesn't like per-cpu accesses such as:

static DEFINE_PER_CPU(struct perf_event *, foo[MAXLEN]);
struct perf_event **bar = __get_cpu_var(foo);

and shouts quite loudly about it:

| warning: incorrect type in assignment (different modifiers)
|    expected struct perf_event **slots
|    got struct perf_event *[noderef] *<noident>

This patch adds casts to these sorts of assignments in hw_breakpoint.c
in order to silence the warnings.
Reported-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Single-stepping a breakpoint requires us to disable it temporarily so that
we don't get stuck in a recursive debug trap. With per-cpu breakpoints this
presents a problem where an interrupt can be taken before the single-step has
completed and a new task is eventually scheduled. This new task will not
hit the breakpoint because it will have been disabled during the previous
handling code.

This patch disallows per-cpu breakpoints on ARM when an overflow handler
is not present. A similar effect can be created by placing breakpoints on
a shell and then running applications there.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The single-stepping code is currently different depending on whether
we are stepping over a breakpoint or a watchpoint. There is no good
reason for this, so let's sort it out.

This patch adds functions for enabling/disabling single-step for
a particular hw_breakpoint and integrates this with the exception
handling code.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The watchpoint single-stepping code calls register_user_hw_breakpoint to
register a mismatch breakpoint for stepping over the watchpoint. This is
performed with preemption disabled, which is unsafe as we may end up scheduling
whilst in_atomic(). Furthermore, using the perf API is rather overkill since
we are already in the hw-breakpoint backend and only require access to reserved
breakpoints anyway.

This patch reworks the watchpoint stepping code so that we don't require
another perf_event for the mismatch breakpoint. Instead, we hold a separate
arch_hw_breakpoint_ctrl struct inside the watchpoint which is used exclusively
for stepping. We can check whether or not stepping is enabled when installing
or uninstalling the watchpoint and operate on the breakpoint accordingly.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

To permit handling of watchpoint exceptions without signalling a
debugger, it is necessary to reserve breakpoint registers for in-kernel
use only.

This patch ensures that we record and subtract the number of reserved
breakpoints from the number of usable breakpoint registers that we
advertise to userspace via the ptrace API.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

On ARM, debug exceptions occur in the form of data or prefetch aborts.
One difference is that debug exceptions require access to per-cpu banked
registers and data structures which are not saved in the low-level exception
code. For kernels built with CONFIG_PREEMPT, there is an unlikely scenario
that the debug handler ends up running on a different CPU from the one
that originally signalled the event, resulting in random data being read
from the wrong registers.

This patch adds a debug_entry macro to the low-level exception handling
code which checks whether the taken exception is a debug exception. If
it is, the preempt count for the faulting process is incremented. After
the debug handler has finished, the count is decremented.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The current hw_breakpoint code tries to fix up the alignment of
breakpoints so that we can make use of sparse byte-address-select
bits in the control register and give the illusion that we can
set breakpoints on unaligned addresses.

Although this works on v6 cores, v7 forbids this behaviour, instead
requiring breakpoints to be set on aligned addresses and have contiguous
byte-address-select ranges depending on the instruction set in use.
For ARM the only supported size is 4 bytes, whilst Thumb-2 also permits
2 byte breakpoints (watchpoints can be of 1, 2, 4 or 8 bytes long).

This patch simplifies the alignment fixup code so that we require
addresses to be aligned to the size of the corresponding breakpoint.
This allows us to handle the common case of breaking on a half-word
aligned Thumb-2 instruction and also allows us to set byte watchpoints
on arbitrary addresses.
Signed-off-by: NWill Deacon <will.deacon@arm.com>