If we are not changing the control register value, avoid writing to it.
Writes to the control register can be very expensive, taking around a
hundred cycles or so.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ARMv6 and greater introduced a new instruction ("bx") which can be used
to return from function calls.  Recent CPUs perform better when the
"bx lr" instruction is used rather than the "mov pc, lr" instruction,
and this sequence is strongly recommended to be used by the ARM
architecture manual (section A.4.1.1).

We provide a new macro "ret" with all its variants for the condition
code which will resolve to the appropriate instruction.

Rather than doing this piecemeal, and miss some instances, change all
the "mov pc" instances to use the new macro, with the exception of
the "movs" instruction and the kprobes code.  This allows us to detect
the "mov pc, lr" case and fix it up - and also gives us the possibility
of deploying this for other registers depending on the CPU selection.
Reported-by: NWill Deacon <will.deacon@arm.com>
Tested-by: Stephen Warren <swarren@nvidia.com> # Tegra Jetson TK1
Tested-by: Robert Jarzmik <robert.jarzmik@free.fr> # mioa701_bootresume.S
Tested-by: Andrew Lunn <andrew@lunn.ch> # Kirkwood
Tested-by: NShawn Guo <shawn.guo@freescale.com>
Tested-by: Tony Lindgren <tony@atomide.com> # OMAPs
Tested-by: Gregory CLEMENT <gregory.clement@free-electrons.com> # Armada XP, 375, 385
Acked-by: Sekhar Nori <nsekhar@ti.com> # DaVinci
Acked-by: Christoffer Dall <christoffer.dall@linaro.org> # kvm/hyp
Acked-by: Haojian Zhuang <haojian.zhuang@gmail.com> # PXA3xx
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> # Xen
Tested-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> # ARMv7M
Tested-by: Simon Horman <horms+renesas@verge.net.au> # Shmobile
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We can use the alignment_trap assembly macro here too.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

cr_no_alignment is really only used by the alignment code.  Since we no
longer change the setting of cr_alignment after boot, we can localise
this to alignment.c
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We will reach fixup handler when one thread(say cpu0) caused an undefined exception, while another thread(say cpu1) is unmmaping the page.

Fixup handler returns to the next userspace instruction which has caused the undef execption, rather than going to the same instruction.

ARM ARM says that after undefined exception, the PC will be pointing
to the next instruction. ie +4 offset in case of ARM and +2 in case of Thumb

And there is no correction offset passed to vector_stub in case of
undef exception.

File: arch/arm/kernel/entry-armv.S +1085
vector_stub     und, UND_MODE

During an undefined exception, in normal scenario(ie when ldrt
instruction does not cause an abort) after resorting the context in
VFP hardware, the PC is modified as show below before jumping to
ret_from_exception which is in r9.

File: arch/arm/vfp/vfphw.S +169
@ The context stored in the VFP hardware is up to date with this thread
vfp_hw_state_valid:
   tst     r1, #FPEXC_EX
   bne     process_exception     @ might as well handle the pending
                                 @ exception before retrying branch
                                 @ out before setting an FPEXC that
                                 @ stops us reading stuff
        VFPFMXR FPEXC, r1        @ Restore FPEXC last
        sub     r2, r2, #4       @ Retry current instruction - if Thumb
        str     r2, [sp, #S_PC]  @ mode it's two 16-bit instructions,
                                 @ else it's one 32-bit instruction, so
                                 @ always subtract 4 from the following
                                 @ instruction address.

But if ldrt results in an abort, we reach the fixup handler and return
to ret_from_execption without correcting the pc.

This patch modifes the fixup handler to re-execute the same instruction which caused undefined execption.
Signed-off-by: NVinayak Menon <vinayakm.list@gmail.com>
Signed-off-by: NArun KS <getarunks@gmail.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The Undef abort handler in the kernel reads the undefined instruction
from user space. If the page table was modified from another CPU, the
user access could fail and do_page_fault() will be executed with
interrupts disabled. This can potentially deadlock on ARM11MPCore or on
Cortex-A15 with erratum 798181 workaround enabled (both implying IPI for
TLB maintenance with page table lock held).

This patch enables the IRQs in __und_usr before attempting to read the
instruction from user space.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Tested-by: NArun KS <getarunks@gmail.com>
Cc: Hartley Sweeten <hsweeten@visionengravers.com>
Cc: Ryan Mallon <rmallon@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

__und_usr_thumb function deals with thumb2 opcodes. In case of BE
image, it needs to byteswap half word thumb2 encoded instructions
before further processing them.

Without this fix BE image user-land thread executing first VFP
instruction encoded in thumb2 fails with SIGILL, because kernel
does not recognize instruction and does not enable VFP.
Reported-by: NCorey Melton <comelton@cisco.com>
Signed-off-by: NVictor Kamensky <victor.kamensky@linaro.org>
Tested-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The exception handling code fails to clear the IT state, potentially
leading to incorrect execution of the fixup if the size of the IT
block is more than one.

Let fixup_exception do the IT sanitizing if a fixup has been found,
and restore CPSR from the stack when returning from a data abort.

Cc: Will Deacon <will.deacon@arm.com>
Cc: stable@vger.kernel.org
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add ARM_BE8() helper to wrap any code conditional on being
compile when CONFIG_ARM_ENDIAN_BE8 is selected and convert
existing places where this is to use it.
Acked-by: NNicolas Pitre <nico@linaro.org>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NBen Dooks <ben.dooks@codethink.co.uk>

Fix yet another build failure caused by a weird set of configuration
settings:

  LD      init/built-in.o
arch/arm/kernel/built-in.o: In function `__dabt_usr':
/home/tom3q/kernel/arch/arm/kernel/entry-armv.S:377: undefined reference to `kuser_cmpxchg64_fixup'
arch/arm/kernel/built-in.o: In function `__irq_usr':
/home/tom3q/kernel/arch/arm/kernel/entry-armv.S:387: undefined reference to `kuser_cmpxchg64_fixup'

caused by:
CONFIG_KUSER_HELPERS=n
CONFIG_CPU_32v6K=n
CONFIG_NEEDS_SYSCALL_FOR_CMPXCHG=n
Reported-by: NTomasz Figa <tomasz.figa@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Provide a kernel configuration option to allow the kernel user helpers
to be removed from the vector page, thereby preventing their use with
ROP (return orientated programming) attacks.  This option is only
visible for CPU architectures which natively support all the operations
which kernel user helpers would normally provide, and must be enabled
with caution.

Cc: <stable@vger.kernel.org>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

FIQ should no longer copy the FIQ code into the user visible vector
page.  Instead, it should use the hidden page.  This change makes
that happen.

Cc: <stable@vger.kernel.org>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Use linker magic to create the vectors and vector stubs: we can tell the
linker to place them at an appropriate VMA, but keep the LMA within the
kernel.  This gets rid of some unnecessary symbol manipulation, and
have the linker calculate the relocations appropriately.

Cc: <stable@vger.kernel.org>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Move the machine vector stubs into the page above the vector page,
which we can prevent from being visible to userspace.  Also move
the reset stub, and place the swi vector at a location that the
'ldr' can get to it.

This hides pointers into the kernel which could give valuable
information to attackers, and reduces the number of exploitable
instructions at a fixed address.

Cc: <stable@vger.kernel.org>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Poison the memory between each kuser helper.  This ensures that any
branch between the kuser helpers will be appropriately trapped.

Cc: <stable@vger.kernel.org>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Since commit 6a1c5312 the user writeable TLS register was zeroed to
prevent it from being used as a covert channel between two tasks.

There are more and more applications coming to Windows RT,
Wine could support them, but mostly they expect to have
the thread environment block (TEB) in TPIDRURW.

This patch preserves that register per thread instead of clearing it.
Unlike the TPIDRURO, which is already switched, the TPIDRURW
can be updated from userspace so needs careful treatment in the case that we
modify TPIDRURW and call fork(). To avoid this we must always read
TPIDRURW in copy_thread.
Signed-off-by: NAndré Hentschel <nerv@dawncrow.de>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJonathan Austin <jonathan.austin@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

commit 91d1aa43 (context_tracking: New context tracking susbsystem)
generalized parts of the RCU userspace extended quiescent state into
the context tracking subsystem.  Context tracking is then used
to implement adaptive tickless (a.k.a extended nohz)

To support the new context tracking subsystem on ARM, the user/kernel
boundary transtions need to be instrumented.

For exceptions and IRQs in usermode, the existing usr_entry macro is
used to instrument the user->kernel transition.  For the return to
usermode path, the ret_to_user* path is instrumented.  Using the
usr_entry macro, this covers interrupts in userspace, data abort and
prefetch abort exceptions in userspace as well as undefined exceptions
in userspace (which is where FP emulation and VFP are handled.)

For syscalls, the slow return path is covered by instrumenting the
ret_to_user path.  In addition, the syscall entry point is
instrumented which covers the user->kernel transition for both fast
and slow syscalls, and an additional instrumentation point is added
for the fast syscall return path (ret_fast_syscall).

Cc: Mats Liljegren <mats.liljegren@enea.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: NKevin Hilman <khilman@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

All svc exit paths need IRQs off.  Rather than placing this before
every user of svc_exit, combine it into this macro.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The IRQ tracing exit path is much the same between all SVC mode
exits, so move this into the svc_exit macro.  Use a macro parameter
to identify the IRQ case, which is the only different case there is.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We don't need to keep reloading the thread into into r10 - we can do
this once and keep the value cached in the register.  Also, schedule
some instructions better so that the pipeline doesn't stall after a
load in the neon code.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

While trying to get a v3.5 kernel booted on the cubox, I noticed that
VFP does not work correctly with VFP bounce handling.  This is because
of the confusion over 16-bit vs 32-bit instructions, and where PC is
supposed to point to.

The rule is that FP handlers are entered with regs->ARM_pc pointing at
the _next_ instruction to be executed.  However, if the exception is
not handled, regs->ARM_pc points at the faulting instruction.

This is easy for ARM mode, because we know that the next instruction and
previous instructions are separated by four bytes.  This is not true of
Thumb2 though.

Since all FP instructions are 32-bit in Thumb2, it makes things easy.
We just need to select the appropriate adjustment.  Do this by moving
the adjustment out of do_undefinstr() into the assembly code, as only
the assembly code knows whether it's dealing with a 32-bit or 16-bit
instruction.

Cc: <stable@vger.kernel.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Fixup entries in the kernel exception tables should be 4-byte aligned
since we return directly to them when handling a faulting instruction in
the kernel.

This patch adds the missing align directives to the fixup entries.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch removes support for ARMv3 CPUs, which haven't worked properly
for quite some time (see the FIXME comment in arch/arm/mm/fault.c).  The
only V3 parts left is the cache model for ARMv3, which is needed for some
odd reason by ARM740T CPUs, and being able to build with -march=armv3,
which is required for the RiscPC platform due to its bus structure.
Acked-by: NWill Deacon <will.deacon@arm.com>
Acked-by: NJean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Disintegrate asm/system.h for ARM.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
cc: Russell King <linux@arm.linux.org.uk>
cc: linux-arm-kernel@lists.infradead.org

Several platforms create IOMEM defines for casting to 'void __iomem *',
and other platforms are incorrectly using __io() macro for the same
purpose. This creates a common definition and removes all the platform
specific versions. Rather than try to make linux/io.h and asm/io.h
assembly safe, the assembly version of IOMEM is moved into
asm/assembler.h.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Sekhar Nori <nsekhar@ti.com>
Cc: Kevin Hilman <khilman@ti.com>
Acked-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Cc: Ryan Mallon <rmallon@gmail.com>
Cc: Eric Miao <eric.y.miao@gmail.com>
Cc: Haojian Zhuang <haojian.zhuang@marvell.com>
Acked-by: NDavid Brown <davidb@codeaurora.org>
Cc: Daniel Walker <dwalker@fifo99.com>
Cc: Bryan Huntsman <bryanh@codeaurora.org>
Cc: Sascha Hauer <kernel@pengutronix.de>
Cc: Shawn Guo <shawn.guo@linaro.org>
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NPaul Walmsley <paul@pwsan.com>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Cc: Rajeev Kumar <rajeev-dlh.kumar@st.com>
Cc: Colin Cross <ccross@android.com>
Cc: Olof Johansson <olof@lixom.net>
Cc: Stephen Warren <swarren@nvidia.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NArnd Bergmann <arnd@arndb.de>

With the removal of disable_fiq on rpc and addition MULTI_IRQ_HANDLER,
entry-macro.S is no longer needed for platforms that select
MULTI_IRQ_HANDLER and the include of it can be conditional.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NH Hartley Sweeten <hsweeten@visionengravers.com>

__kuser_cmpxchg64 has a return path using bx lr to get back to the caller.
This is actually ok since the code in question is predicated on
CONFIG_CPU_32v6K, but for the sake of consistency using the usr_ret
macro is probably better.
Acked-by: NDave Martin <dave.martin@linaro.org>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Fix compilation failure, when Thumb support is not enabled:

arch/arm/kernel/entry-armv.S: Assembler messages:
arch/arm/kernel/entry-armv.S:501: Error: backward ref to unknown label "2:"
arch/arm/kernel/entry-armv.S:502: Error: backward ref to unknown label "3:"
make[2]: *** [arch/arm/kernel/entry-armv.o] Error 1
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Reviewed-by: NDave Martin <dave.martin@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Even when CONFIG_MULTI_IRQ_HANDLER is selected, the core code
requires the arch_irq_handler_default macro to be defined as
a fallback.

It turns out nobody is using that particular feature as both PXA
and shmobile have all their machine descriptors populated with
the interrupt handler, leaving unused code (or empty macros) in
their entry-macro.S file just to be able to compile entry-armv.S.

Make CONFIG_MULTI_IRQ_HANDLER exclusive wrt arch_irq_handler_default,
which allows to remove one test from the hot path. Also cleanup both
PXA and shmobile entry-macro.S.

Cc: Paul Mundt <lethal@linux-sh.org>
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NEric Miao <eric.y.miao@gmail.com>
Tested-by: NJamie Iles <jamie@jamieiles.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

When v6 and >=v7 boards are supported in the same kernel, the
__und_usr code currently makes a build-time assumption that Thumb-2
instructions occurring in userspace don't need to be supported.
Strictly speaking this is incorrect.

This patch fixes the above case by doing a run-time check on the
CPU architecture in these cases.  This only affects kernels which
support v6 and >=v7 CPUs together: plain v6 and plain v7 kernels
are unaffected.
Signed-off-by: NDave Martin <dave.martin@linaro.org>
Reviewed-by: NJon Medhurst <tixy@yxit.co.uk>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

When testing whether a Thumb-2 instruction is 32 bits long or not,
the masking done in order to test bits 11-15 of the first
instruction halfword won't affect the result of the comparison, so
remove it.
Signed-off-by: NDave Martin <dave.martin@linaro.org>
Reviewed-by: NJon Medhurst <tixy@yxit.co.uk>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The SVC IRQ, prefetch and data abort handlers preserve the SPSR value
via r5 across the exception.  Rather than re-loading it from pt_regs,
use the preserved value instead.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Tail-call the main C data abort handler code from the per-CPU helper
code.  Update the comments in the code wrt the new calling and return
register state.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Re-jig the CPU abort helpers to take the PC/PSR in r4/r5 rather
than r2/r3.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Tail-call the main C prefetch abort handler code from the per-CPU
helper code.  Also note that the helper function becomes ABI
compliant in terms of the registers preserved.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This avoids the irq entry assembly corrupting r5, thereby allowing it
to be preserved through to the svc exit code.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

All handlers now call trace_hardirqs_off, so move this common code into
the (svc|usr)_entry assembler macros.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

As we no longer re-enable interrupts in these exception handlers, add
the irqsoff tracing calls to them so that the kernel tracks the state
more accurately.

Note that these calls are conditional on IRQSOFF_TRACER:

  kernel ----------> user ---------> kernel
          ^ irqs enabled   ^ irqs disabled

No kernel code can run on the local CPU until we've re-entered the
kernel through one of the exception handlers - and userspace can not
take any locks etc.  So, the kernel doesn't care about the IRQ mask
state while userspace is running unless we're doing IRQ off latency
tracing.  So, we can (and do) avoid the overhead of updating the IRQ
mask state on every kernel->user and user->kernel transition.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add irqtrace function calls to the undefined exception handler, so
that we get sane lockdep traces from locking problems in undefined
exception handlers.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Avoid enabling interrupts if the parent context had interrupts enabled
in the abort handler assembly code, and move this into the breakpoint/
page/alignment fault handlers instead.

This gets rid of some special-casing for the breakpoint fault handlers
from the low level abort handler path.
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>