Since commit 6a1c5312 the user writeable TLS register was zeroed to
prevent it from being used as a covert channel between two tasks.

There are more and more applications coming to Windows RT,
Wine could support them, but mostly they expect to have
the thread environment block (TEB) in TPIDRURW.

This patch preserves that register per thread instead of clearing it.
Unlike the TPIDRURO, which is already switched, the TPIDRURW
can be updated from userspace so needs careful treatment in the case that we
modify TPIDRURW and call fork(). To avoid this we must always read
TPIDRURW in copy_thread.
Signed-off-by: NAndré Hentschel <nerv@dawncrow.de>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJonathan Austin <jonathan.austin@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

commit 91d1aa43 (context_tracking: New context tracking susbsystem)
generalized parts of the RCU userspace extended quiescent state into
the context tracking subsystem.  Context tracking is then used
to implement adaptive tickless (a.k.a extended nohz)

To support the new context tracking subsystem on ARM, the user/kernel
boundary transtions need to be instrumented.

For exceptions and IRQs in usermode, the existing usr_entry macro is
used to instrument the user->kernel transition.  For the return to
usermode path, the ret_to_user* path is instrumented.  Using the
usr_entry macro, this covers interrupts in userspace, data abort and
prefetch abort exceptions in userspace as well as undefined exceptions
in userspace (which is where FP emulation and VFP are handled.)

For syscalls, the slow return path is covered by instrumenting the
ret_to_user path.  In addition, the syscall entry point is
instrumented which covers the user->kernel transition for both fast
and slow syscalls, and an additional instrumentation point is added
for the fast syscall return path (ret_fast_syscall).

Cc: Mats Liljegren <mats.liljegren@enea.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: NKevin Hilman <khilman@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

All svc exit paths need IRQs off.  Rather than placing this before
every user of svc_exit, combine it into this macro.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The IRQ tracing exit path is much the same between all SVC mode
exits, so move this into the svc_exit macro.  Use a macro parameter
to identify the IRQ case, which is the only different case there is.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We don't need to keep reloading the thread into into r10 - we can do
this once and keep the value cached in the register.  Also, schedule
some instructions better so that the pipeline doesn't stall after a
load in the neon code.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

While trying to get a v3.5 kernel booted on the cubox, I noticed that
VFP does not work correctly with VFP bounce handling.  This is because
of the confusion over 16-bit vs 32-bit instructions, and where PC is
supposed to point to.

The rule is that FP handlers are entered with regs->ARM_pc pointing at
the _next_ instruction to be executed.  However, if the exception is
not handled, regs->ARM_pc points at the faulting instruction.

This is easy for ARM mode, because we know that the next instruction and
previous instructions are separated by four bytes.  This is not true of
Thumb2 though.

Since all FP instructions are 32-bit in Thumb2, it makes things easy.
We just need to select the appropriate adjustment.  Do this by moving
the adjustment out of do_undefinstr() into the assembly code, as only
the assembly code knows whether it's dealing with a 32-bit or 16-bit
instruction.

Cc: <stable@vger.kernel.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Fixup entries in the kernel exception tables should be 4-byte aligned
since we return directly to them when handling a faulting instruction in
the kernel.

This patch adds the missing align directives to the fixup entries.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch removes support for ARMv3 CPUs, which haven't worked properly
for quite some time (see the FIXME comment in arch/arm/mm/fault.c).  The
only V3 parts left is the cache model for ARMv3, which is needed for some
odd reason by ARM740T CPUs, and being able to build with -march=armv3,
which is required for the RiscPC platform due to its bus structure.
Acked-by: NWill Deacon <will.deacon@arm.com>
Acked-by: NJean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Disintegrate asm/system.h for ARM.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
cc: Russell King <linux@arm.linux.org.uk>
cc: linux-arm-kernel@lists.infradead.org

Several platforms create IOMEM defines for casting to 'void __iomem *',
and other platforms are incorrectly using __io() macro for the same
purpose. This creates a common definition and removes all the platform
specific versions. Rather than try to make linux/io.h and asm/io.h
assembly safe, the assembly version of IOMEM is moved into
asm/assembler.h.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Sekhar Nori <nsekhar@ti.com>
Cc: Kevin Hilman <khilman@ti.com>
Acked-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Cc: Ryan Mallon <rmallon@gmail.com>
Cc: Eric Miao <eric.y.miao@gmail.com>
Cc: Haojian Zhuang <haojian.zhuang@marvell.com>
Acked-by: NDavid Brown <davidb@codeaurora.org>
Cc: Daniel Walker <dwalker@fifo99.com>
Cc: Bryan Huntsman <bryanh@codeaurora.org>
Cc: Sascha Hauer <kernel@pengutronix.de>
Cc: Shawn Guo <shawn.guo@linaro.org>
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NPaul Walmsley <paul@pwsan.com>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Cc: Rajeev Kumar <rajeev-dlh.kumar@st.com>
Cc: Colin Cross <ccross@android.com>
Cc: Olof Johansson <olof@lixom.net>
Cc: Stephen Warren <swarren@nvidia.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NArnd Bergmann <arnd@arndb.de>

With the removal of disable_fiq on rpc and addition MULTI_IRQ_HANDLER,
entry-macro.S is no longer needed for platforms that select
MULTI_IRQ_HANDLER and the include of it can be conditional.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NH Hartley Sweeten <hsweeten@visionengravers.com>

__kuser_cmpxchg64 has a return path using bx lr to get back to the caller.
This is actually ok since the code in question is predicated on
CONFIG_CPU_32v6K, but for the sake of consistency using the usr_ret
macro is probably better.
Acked-by: NDave Martin <dave.martin@linaro.org>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Fix compilation failure, when Thumb support is not enabled:

arch/arm/kernel/entry-armv.S: Assembler messages:
arch/arm/kernel/entry-armv.S:501: Error: backward ref to unknown label "2:"
arch/arm/kernel/entry-armv.S:502: Error: backward ref to unknown label "3:"
make[2]: *** [arch/arm/kernel/entry-armv.o] Error 1
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Reviewed-by: NDave Martin <dave.martin@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Even when CONFIG_MULTI_IRQ_HANDLER is selected, the core code
requires the arch_irq_handler_default macro to be defined as
a fallback.

It turns out nobody is using that particular feature as both PXA
and shmobile have all their machine descriptors populated with
the interrupt handler, leaving unused code (or empty macros) in
their entry-macro.S file just to be able to compile entry-armv.S.

Make CONFIG_MULTI_IRQ_HANDLER exclusive wrt arch_irq_handler_default,
which allows to remove one test from the hot path. Also cleanup both
PXA and shmobile entry-macro.S.

Cc: Paul Mundt <lethal@linux-sh.org>
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NEric Miao <eric.y.miao@gmail.com>
Tested-by: NJamie Iles <jamie@jamieiles.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

When v6 and >=v7 boards are supported in the same kernel, the
__und_usr code currently makes a build-time assumption that Thumb-2
instructions occurring in userspace don't need to be supported.
Strictly speaking this is incorrect.

This patch fixes the above case by doing a run-time check on the
CPU architecture in these cases.  This only affects kernels which
support v6 and >=v7 CPUs together: plain v6 and plain v7 kernels
are unaffected.
Signed-off-by: NDave Martin <dave.martin@linaro.org>
Reviewed-by: NJon Medhurst <tixy@yxit.co.uk>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

When testing whether a Thumb-2 instruction is 32 bits long or not,
the masking done in order to test bits 11-15 of the first
instruction halfword won't affect the result of the comparison, so
remove it.
Signed-off-by: NDave Martin <dave.martin@linaro.org>
Reviewed-by: NJon Medhurst <tixy@yxit.co.uk>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The SVC IRQ, prefetch and data abort handlers preserve the SPSR value
via r5 across the exception.  Rather than re-loading it from pt_regs,
use the preserved value instead.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Tail-call the main C data abort handler code from the per-CPU helper
code.  Update the comments in the code wrt the new calling and return
register state.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Re-jig the CPU abort helpers to take the PC/PSR in r4/r5 rather
than r2/r3.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Tail-call the main C prefetch abort handler code from the per-CPU
helper code.  Also note that the helper function becomes ABI
compliant in terms of the registers preserved.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This avoids the irq entry assembly corrupting r5, thereby allowing it
to be preserved through to the svc exit code.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

All handlers now call trace_hardirqs_off, so move this common code into
the (svc|usr)_entry assembler macros.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

As we no longer re-enable interrupts in these exception handlers, add
the irqsoff tracing calls to them so that the kernel tracks the state
more accurately.

Note that these calls are conditional on IRQSOFF_TRACER:

  kernel ----------> user ---------> kernel
          ^ irqs enabled   ^ irqs disabled

No kernel code can run on the local CPU until we've re-entered the
kernel through one of the exception handlers - and userspace can not
take any locks etc.  So, the kernel doesn't care about the IRQ mask
state while userspace is running unless we're doing IRQ off latency
tracing.  So, we can (and do) avoid the overhead of updating the IRQ
mask state on every kernel->user and user->kernel transition.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add irqtrace function calls to the undefined exception handler, so
that we get sane lockdep traces from locking problems in undefined
exception handlers.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Avoid enabling interrupts if the parent context had interrupts enabled
in the abort handler assembly code, and move this into the breakpoint/
page/alignment fault handlers instead.

This gets rid of some special-casing for the breakpoint fault handlers
from the low level abort handler path.
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This avoids unnecessary instructions for CPUs which implement the IFAR
(instruction fault address register).
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This allows us to avoid moving registers twice to work around the
clobbered registers when we add calls to trace_hardirqs_{on,off}.

Ensure that all SVC handlers return with SPSR in r5 for consistency.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

There's no point checking to see whether IRQs were masked in the parent
context when returning from IRQ handling - the fact that we're handling
an IRQ means that the parent context must have had IRQs unmasked.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

irq_enter() and irq_exit() already take care of the preempt_count
handling for interrupts, which increment and decrement the hardirq
bits of the preempt count.  So we can remove the preempt count handing
in our IRQ entry/exit assembly, like x86 did some 9 years ago.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Replace r4 with ip for calling abort helpers - ip is allowed to be
corrupted by called functions in the ABI, so it makes more sense to
use such a register.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Some user space applications are designed around the ability to perform
atomic operations on 64 bit values.  Since this is natively possible
only with ARMv6k and above, let's provide a new kuser helper to perform
the operation with kernel supervision on pre ARMv6k hardware.
Signed-off-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Tested-by: NDave Martin <dave.martin@linaro.org>

Digging into some assembly file in order to get information about the
kuser helpers is not that convivial.  Let's move that information to
a better formatted file in Documentation/arm/ and improve on it a bit.

Thanks to Dave Martin <dave.martin@linaro.org> for the initial cleanup and
clarifications.
Signed-off-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Acked-by: NDave Martin <dave.martin@linaro.org>

This patch fixes the lockdep warning of "unannotated irqs-off"[1].

After entering __irq_usr, arm core will disable interrupt automatically,
but __irq_usr does not annotate the irq disable, so lockdep may complain
the warning if it has chance to check this in irq handler.

This patch adds trace_hardirqs_off in __irq_usr before entering irq_handler
to handle the irq, also calls ret_to_user_from_irq to avoid calling
disable_irq again.

This is also a fix for irq off tracer.

[1], lockdep warning log of "unannotated irqs-off"

[   13.804687] ------------[ cut here ]------------
[   13.809570] WARNING: at kernel/lockdep.c:3335 check_flags+0x78/0x1d0()
[   13.816467] Modules linked in:
[   13.819732] Backtrace:
[   13.822357] [<c01cb42c>] (dump_backtrace+0x0/0x100) from [<c06abb14>] (dump_stack+0x20/0x24)
[   13.831268]  r6:c07d8c2c r5:00000d07 r4:00000000 r3:00000000
[   13.837280] [<c06abaf4>] (dump_stack+0x0/0x24) from [<c01ffc04>] (warn_slowpath_common+0x5c/0x74)
[   13.846649] [<c01ffba8>] (warn_slowpath_common+0x0/0x74) from [<c01ffc48>] (warn_slowpath_null+0x2c/0x34)
[   13.856781]  r8:00000000 r7:00000000 r6:c18b8194 r5:60000093 r4:ef182000
[   13.863708] r3:00000009
[   13.866485] [<c01ffc1c>] (warn_slowpath_null+0x0/0x34) from [<c0237d84>] (check_flags+0x78/0x1d0)
[   13.875823] [<c0237d0c>] (check_flags+0x0/0x1d0) from [<c023afc8>] (lock_acquire+0x4c/0x150)
[   13.884704] [<c023af7c>] (lock_acquire+0x0/0x150) from [<c06af638>] (_raw_spin_lock+0x4c/0x84)
[   13.893798] [<c06af5ec>] (_raw_spin_lock+0x0/0x84) from [<c01f9a44>] (sched_ttwu_pending+0x58/0x8c)
[   13.903320]  r6:ef92d040 r5:00000003 r4:c18b8180
[   13.908233] [<c01f99ec>] (sched_ttwu_pending+0x0/0x8c) from [<c01f9a90>] (scheduler_ipi+0x18/0x1c)
[   13.917663]  r6:ef183fb0 r5:00000003 r4:00000000 r3:00000001
[   13.923645] [<c01f9a78>] (scheduler_ipi+0x0/0x1c) from [<c01bc458>] (do_IPI+0x9c/0xfc)
[   13.932006] [<c01bc3bc>] (do_IPI+0x0/0xfc) from [<c06b0888>] (__irq_usr+0x48/0xe0)
[   13.939971] Exception stack(0xef183fb0 to 0xef183ff8)
[   13.945281] 3fa0:                                     ffffffc3 0001500c 00000001 0001500c
[   13.953948] 3fc0: 00000050 400b45f0 400d9000 00000000 00000001 400d9600 6474e552 bea05b3c
[   13.962585] 3fe0: 400d96c0 bea059c0 400b6574 400b65d8 20000010 ffffffff
[   13.969573]  r6:00000403 r5:fa240100 r4:ffffffff r3:20000010
[   13.975585] ---[ end trace efc4896ab0fb62cb ]---
[   13.980468] possible reason: unannotated irqs-off.
[   13.985534] irq event stamp: 1610
[   13.989044] hardirqs last  enabled at (1610): [<c01c703c>] no_work_pending+0x8/0x2c
[   13.997131] hardirqs last disabled at (1609): [<c01c7024>] ret_slow_syscall+0xc/0x1c
[   14.005371] softirqs last  enabled at (0): [<c01fe5e4>] copy_process+0x2cc/0xa24
[   14.013183] softirqs last disabled at (0): [<  (null)>]   (null)
Signed-off-by: NMing Lei <ming.lei@canonical.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This allows the cache/processor/fault glue to be more easily used
from assembler code.  Tested on Assabet and Tegra 2.
Tested-by: NColin Cross <ccross@android.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Per subarch interrupt handler macros V3.

This patch breaks out code from the irq_handler macro
into arch_irq_handler and arch_irq_handler_default.

The macros are put in the header file "entry-macro-multi.S"

The arch_irq_handler_default macro is designed to be
used by irq_handler in entry-armv.S while arch_irq_handler
is suitable for per-subarch use.
Signed-off-by: NMagnus Damm <damm@opensource.se>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Normally different ARM platform has different way to decode the IRQ
hardware status and demultiplex to the corresponding IRQ handler.
This is highly optimized by macro irq_handler in entry-armv.S, and
each machine defines their own macro to decode the IRQ number.
However, this prevents multiple machine classes to be built into a
single kernel.

By allowing each machine to specify thier own handler, and making
function pointer 'handle_arch_irq' to point to it at run time, this
can be solved. And introduce CONFIG_MULTI_IRQ_HANDLER to allow both
solutions to work.

Comparing with the highly optimized macro of irq_handler, the new
function must be written with care not to lose too much performance.
And the IPI stuff on SMP is expected to move to the provided arch
IRQ handler as well.

The assembly code to invoke handle_arch_irq is optimized by Russell
King.
Signed-off-by: NEric Miao <eric.miao@canonical.com>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

  * __fixup_smp_on_up has been modified with support for the
    THUMB2_KERNEL case.  For THUMB2_KERNEL only, fixups are split
    into halfwords in case of misalignment, since we can't rely on
    unaligned accesses working before turning the MMU on.

    No attempt is made to optimise the aligned case, since the
    number of fixups is typically small, and it seems best to keep
    the code as simple as possible.

  * Add a rotate in the fixup_smp code in order to support
    CPU_BIG_ENDIAN, as suggested by Nicolas Pitre.

  * Add an assembly-time sanity-check to ALT_UP() to ensure that
    the content really is the right size (4 bytes).

    (No check is done for ALT_SMP().  Possibly, this could be fixed
    by splitting the two uses ot ALT_SMP() (ALT_SMP...SMP_UP versus
    ALT_SMP...SMP_UP_B) into two macros.  In the first case,
    ALT_SMP needs to expand to >= 4 bytes, not == 4.)

  * smp_mpidr.h (which implements ALT_SMP()/ALT_UP() manually due
    to macro limitations) has not been modified: the affected
    instruction (mov) has no 16-bit encoding, so the correct
    instruction size is satisfied in this case.

  * A "mode" parameter has been added to smp_dmb:

    smp_dmb arm @ assumes 4-byte instructions (for ARM code, e.g. kuser)
    smp_dmb     @ uses W() to ensure 4-byte instructions for ALT_SMP()

    This avoids assembly failures due to use of W() inside smp_dmb,
    when assembling pure-ARM code in the vectors page.

    There might be a better way to achieve this.

  * Kconfig: make SMP_ON_UP depend on
    (!THUMB2_KERNEL || !BIG_ENDIAN) i.e., THUMB2_KERNEL is now
    supported, but only if !BIG_ENDIAN (The fixup code for Thumb-2
    currently assumes little-endian order.)

Tested using a single generic realview kernel on:
	ARM RealView PB-A8 (CONFIG_THUMB2_KERNEL={n,y})
	ARM RealView PBX-A9 (SMP)
Signed-off-by: NDave Martin <dave.martin@linaro.org>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

On ARM, debug exceptions occur in the form of data or prefetch aborts.
One difference is that debug exceptions require access to per-cpu banked
registers and data structures which are not saved in the low-level exception
code. For kernels built with CONFIG_PREEMPT, there is an unlikely scenario
that the debug handler ends up running on a different CPU from the one
that originally signalled the event, resulting in random data being read
from the wrong registers.

This patch adds a debug_entry macro to the low-level exception handling
code which checks whether the taken exception is a debug exception. If
it is, the preempt count for the faulting process is incremented. After
the debug handler has finished, the count is decremented.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The existing code invokes the syscall with rubbish in r7,
due to what looks like an incorrect literal load idiom.
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NDave Martin <dave.martin@linaro.org>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>