Provide hooks into the kernel entry and exit paths to permit control
of userspace visibility to the kernel.  The intended use is:

- on entry to kernel from user, uaccess_disable will be called to
  disable userspace visibility
- on exit from kernel to user, uaccess_enable will be called to
  enable userspace visibility
- on entry from a kernel exception, uaccess_save_and_disable will be
  called to save the current userspace visibility setting, and disable
  access
- on exit from a kernel exception, uaccess_restore will be called to
  restore the userspace visibility as it was before the exception
  occurred.

These hooks allows us to keep userspace visibility disabled for the
vast majority of the kernel, except for localised regions where we
want to explicitly access userspace.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The following structure is just asking for trouble:

 #ifdef CONFIG_symbol
	.macro foo
	...
	.endm
	.macro bar
	...
	.endm
	.macro baz
	...
	.endm
 #else
	.macro foo
	...
	.endm
	.macro bar
	...
	.endm
 #ifdef CONFIG_symbol2
	.macro baz
	...
	.endm
 #else
	.macro baz
	...
	.endm
 #endif
 #endif

such as one defintion being updated, but the other definitions miss out.
Where the contents of a macro needs to be conditional, the hint is in
the first clause of this very sentence.  "contents" "conditional".  Not
multiple separate definitions, especially not when much of the macro
is the same between different configs.

This patch fixes this bad style, which had caused the Thumb2 code to
miss-out on the uaccess updates.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Currently restore_user_regs deallocates the SVC stack early in
its execution and relies on no exception being taken between
the deallocation and the registers being restored. The introduction
of a default FIQ handler that also uses the SVC stack breaks this
assumption and can result in corrupted register state.

This patch works around the problem by removing the early
stack deallocation and using r2 as a temporary instead. I have
not found a way to do this without introducing an extra mov
instruction to the macro.
Signed-off-by: NDaniel Thompson <daniel.thompson@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

If we are not changing the control register value, avoid writing to it.
Writes to the control register can be very expensive, taking around a
hundred cycles or so.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch introduces a new default FIQ handler that is structured in a
similar way to the existing ARM exception handler and result in the FIQ
being handled by C code running on the SVC stack (despite this code run
in the FIQ handler is subject to severe limitations with respect to
locking making normal interaction with the kernel impossible).

This default handler allows concepts that on x86 would be handled using
NMIs to be realized on ARM.

Credit:

    This patch is a near complete re-write of a patch originally
    provided by Anton Vorontsov. Today only a couple of small fragments
    survive, however without Anton's work to build from this patch would
    not exist. Thanks also to Russell King for spoonfeeding me a variety
    of fixes during the review cycle.
Signed-off-by: NDaniel Thompson <daniel.thompson@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

On revisions of Cortex-A15 prior to r3p3, a CLREX instruction at PL1 may
falsely trigger a watchpoint exception, leading to potential data aborts
during exception return and/or livelock.

This patch resolves the issue in the following ways:

  - Replacing our uses of CLREX with a dummy STREX sequence instead (as
    we did for v6 CPUs).

  - Removing the clrex code from v7_exit_coherency_flush and derivatives,
    since this only exists as a minor performance improvement when
    non-cached exclusives are in use (Linux doesn't use these).

Benchmarking on a variety of ARM cores revealed no measurable
performance difference with this change applied, so the change is
performed unconditionally and no new Kconfig entry is added.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Cc: stable@vger.kernel.org
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ARMv6 and greater introduced a new instruction ("bx") which can be used
to return from function calls.  Recent CPUs perform better when the
"bx lr" instruction is used rather than the "mov pc, lr" instruction,
and this sequence is strongly recommended to be used by the ARM
architecture manual (section A.4.1.1).

We provide a new macro "ret" with all its variants for the condition
code which will resolve to the appropriate instruction.

Rather than doing this piecemeal, and miss some instances, change all
the "mov pc" instances to use the new macro, with the exception of
the "movs" instruction and the kprobes code.  This allows us to detect
the "mov pc, lr" case and fix it up - and also gives us the possibility
of deploying this for other registers depending on the CPU selection.
Reported-by: NWill Deacon <will.deacon@arm.com>
Tested-by: Stephen Warren <swarren@nvidia.com> # Tegra Jetson TK1
Tested-by: Robert Jarzmik <robert.jarzmik@free.fr> # mioa701_bootresume.S
Tested-by: Andrew Lunn <andrew@lunn.ch> # Kirkwood
Tested-by: NShawn Guo <shawn.guo@freescale.com>
Tested-by: Tony Lindgren <tony@atomide.com> # OMAPs
Tested-by: Gregory CLEMENT <gregory.clement@free-electrons.com> # Armada XP, 375, 385
Acked-by: Sekhar Nori <nsekhar@ti.com> # DaVinci
Acked-by: Christoffer Dall <christoffer.dall@linaro.org> # kvm/hyp
Acked-by: Haojian Zhuang <haojian.zhuang@gmail.com> # PXA3xx
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> # Xen
Tested-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> # ARMv7M
Tested-by: Simon Horman <horms+renesas@verge.net.au> # Shmobile
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We can use the alignment_trap assembly macro here too.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

According to the ARM ARM, the behaviour is UNPREDICTABLE if the PC read
from the exception return stack is not half word aligned.  See the
pseudo code for ExceptionReturn() and PopStack().

The signal handler's address has the bit 0 set, and setup_return()
directly writes this to regs->ARM_pc.  Current hardware happens to
discard this bit, but QEMU's emulation doesn't and this makes processes
crash.  Mask out bit 0 before the exception return in order to get
predictable behaviour.

Fixes: 19c4d593 ("ARM: ARMv7-M: Add support for exception handling")

Cc: stable@kernel.org
Acked-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: NRabin Vincent <rabin@rab.in>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

asm/assembler.h is a better place for this macro since it is used by
asm files outside arch/arm/kernel/
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Tested-by: NArun KS <getarunks@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ad65782f (context_tracking: Optimize main APIs off case
with static key) converted context tracking main APIs to inline
function and left ARM asm callers behind.

This can be easily fixed by making ARM calling the post static
keys context tracking function. We just need to replicate the
static key checks there. We'll remove these later when ARM will
support the context tracking static keys.
Reported-by: NGuenter Roeck <linux@roeck-us.net>
Reported-by: NRussell King <linux@arm.linux.org.uk>
Signed-off-by: NFrederic Weisbecker <fweisbec@gmail.com>
Tested-by: NKevin Hilman <khilman@linaro.org>
Cc: Nicolas Pitre <nicolas.pitre@linaro.org>
Cc: Anil Kumar <anilk4.v@gmail.com>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Benoit Cousson <b-cousson@ti.com>
Cc: Guenter Roeck <linux@roeck-us.net>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Kevin Hilman <khilman@linaro.org>

This patch implements the exception handling for the ARMv7-M
architecture (pretty different from the A or R profiles).

It bases on work done earlier by Catalin for 2.6.33 but was nearly
completely rewritten to use a pt_regs layout compatible to the A
profile.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Reviewed-by: NJonathan Austin <jonathan.austin@arm.com>
Tested-by: NJonathan Austin <jonathan.austin@arm.com>
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>

commit 91d1aa43 (context_tracking: New context tracking susbsystem)
generalized parts of the RCU userspace extended quiescent state into
the context tracking subsystem.  Context tracking is then used
to implement adaptive tickless (a.k.a extended nohz)

To support the new context tracking subsystem on ARM, the user/kernel
boundary transtions need to be instrumented.

For exceptions and IRQs in usermode, the existing usr_entry macro is
used to instrument the user->kernel transition.  For the return to
usermode path, the ret_to_user* path is instrumented.  Using the
usr_entry macro, this covers interrupts in userspace, data abort and
prefetch abort exceptions in userspace as well as undefined exceptions
in userspace (which is where FP emulation and VFP are handled.)

For syscalls, the slow return path is covered by instrumenting the
ret_to_user path.  In addition, the syscall entry point is
instrumented which covers the user->kernel transition for both fast
and slow syscalls, and an additional instrumentation point is added
for the fast syscall return path (ret_fast_syscall).

Cc: Mats Liljegren <mats.liljegren@enea.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: NKevin Hilman <khilman@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

All svc exit paths need IRQs off.  Rather than placing this before
every user of svc_exit, combine it into this macro.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The IRQ tracing exit path is much the same between all SVC mode
exits, so move this into the svc_exit macro.  Use a macro parameter
to identify the IRQ case, which is the only different case there is.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The implementation of svc_exit didn't take into account any stack hole
created by svc_entry; as happens with the undef handler when kprobes are
configured. The fix is to read the saved value of SP rather than trying
to calculate it.
Signed-off-by: NJon Medhurst <tixy@yxit.co.uk>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>

Avoid enabling interrupts if the parent context had interrupts enabled
in the abort handler assembly code, and move this into the breakpoint/
page/alignment fault handlers instead.

This gets rid of some special-casing for the breakpoint fault handlers
from the low level abort handler path.
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

If CONFIG_CPU_V6 is enabled, then the kernel must support ARMv6 CPUs
which don't have the V6K extensions implemented.  Always use the
dummy store-exclusive method to ensure that the exclusive monitors are
cleared.

If CONFIG_CPU_V6 is not set, but CONFIG_CPU_32v6K is enabled, then we
have the K extensions available on all CPUs we're building support for,
so we can use the new clear-exclusive instruction.
Acked-by: NTony Lindgren <tony@atomide.com>
Tested-by: NSourav Poddar <sourav.poddar@ti.com>
Tested-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

On ARM, debug exceptions occur in the form of data or prefetch aborts.
One difference is that debug exceptions require access to per-cpu banked
registers and data structures which are not saved in the low-level exception
code. For kernels built with CONFIG_PREEMPT, there is an unlikely scenario
that the debug handler ends up running on a different CPU from the one
that originally signalled the event, resulting in random data being read
from the wrong registers.

This patch adds a debug_entry macro to the low-level exception handling
code which checks whether the taken exception is a debug exception. If
it is, the preempt count for the faulting process is incremented. After
the debug handler has finished, the count is decremented.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

ARMv5T and earlier require that a ldm {}^ instruction is not followed
by an instruction that accesses banked registers. This patch restores
the nop that was lost in commit b86040a5.
Signed-off-by: NAnders Grafström <grfstrm@users.sourceforge.net>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The 32-bit wide variant of "mov pc, reg" in Thumb-2 is unpredictable
causing improper handling of the undefined instructions not caught by
the kernel. This patch adds a movw_pc macro for such situations
(currently only used in call_fpe).
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Commit 200b812d "Clear the exclusive monitor when returning from an
exception" broke the vast majority of ARM systems in the wild which are
still pre ARMv6.  The kernel is crashing on the first occurrence of an
exception due to the removal of the actual return instruction for them.
Let's add it back.
Signed-off-by: NNicolas Pitre <nico@marvell.com>
Acked-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The patch adds a CLREX or dummy STREX to the exception return path. This
is needed because several atomic/locking operations use a pair of
LDREX/STREXEQ and the EQ condition may not always be satisfied. This
would leave the exclusive monitor status set and may cause problems with
atomic/locking operations in the interrupted code.

With this patch, the atomic_set() operation can be a simple STR
instruction (on SMP systems, the global exclusive monitor is cleared by
STR anyway). Clearing the exclusive monitor during context switch is no
longer needed as this is handled by the exception return path anyway.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Reported-by: NJamie Lokier <jamie@shareable.org>

This patch implements the ARM/Thumb-2 unified kernel start-up and
exception handling code.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Signed-off-by: NJörn Engel <joern@wohnheim.fh-wedel.de>
Signed-off-by: NAdrian Bunk <bunk@stusta.de>

5d25ac03 broke VFP builds due to
enable_irq not being defined as an assembly macro.  Move it to
assembler.h so everyone can use it.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Patch from Nicolas Pitre

The ARM EABI says that the stack pointer has to be 64-bit aligned for
reasons already mentioned in patch #3101 when calling C functions.

We therefore must verify and adjust sp accordingly when taking an
exception from kernel mode since sp might not necessarily be 64-bit
aligned if the exception occurs in the middle of a kernel function.

If the exception occurs while in user mode then no sp fixup is needed as
long as sizeof(struct pt_regs) as well as any additional syscall data
stack space remain multiples of 8.
Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Delete obsoleted stuff from arch Makefile and rename
constants.h to asm-offsets.h
Signed-off-by: NSam Ravnborg <sam@ravnborg.org>

The current vector entry system does not allow for SMP.  In
order to work around this, we need to eliminate our reliance
on the fixed save areas, which breaks the way we enable
alignment traps.  This patch makes the alignment trap enable
code independent of the way we handle the save areas.
Signed-off-by: NRussell King <rmk@arm.linux.org.uk>

Move common includes to entry-header, and file specific includes
to the relevant file.
Signed-off-by: NRussell King <rmk@arm.linux.org.uk>

Assembly macros are pointless if they're only used once.  Move
them inline.
Signed-off-by: NRussell King <rmk@arm.linux.org.uk>

PT_TRACESYS is unused, remove it.
Signed-off-by: NRussell King <rmk@arm.linux.org.uk>

SVC_MODE reflects the MODE_SVC definition in asm/ptrace.h.  Use
the asm/ptrace.h definition instead, and remove SVC_MODE.
Signed-off-by: NRussell King <rmk@arm.linux.org.uk>

Don't define our own local constants, but use those already defined
in asm/unistd.h instead.
Signed-off-by: NRussell King <rmk@arm.linux.org.uk>

Generate pt_regs S_xx offsets from the structure itself instead
of #defining them.
Signed-off-by: NRussell King <rmk@arm.linux.org.uk>

Since we do not require a register for these operations, we can
remove this unnecessary argument.
Signed-off-by: NRussell King <rmk@arm.linux.org.uk>

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!