Before this patch enabling and disabling irqs in assembler code and by
the hardware wasn't tracked completly.

I had to transpose two instructions in arch/arm/lib/bitops.h because
restore_irqs doesn't preserve the flags with CONFIG_TRACE_IRQFLAGS=y
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>

This patch implements the ARM/Thumb-2 unified kernel start-up and
exception handling code.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Since the Thumb-2 instructions can be 16-bit wide, data in the .text
sections may not be aligned to a 32-bit word and this leads to unaligned
exceptions. This patch does not affect the ARM code generation.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Starting with ARMv6, the CPUs support the BE-8 variant of big-endian
(byte-invariant). This patch adds the core support:

- setting of the BE-8 mode via the CPSR.E register for both kernel and
  user threads
- big-endian page table walking
- REV used to rotate instructions read from memory during fault
  processing as they are still little-endian format
- Kconfig and Makefile support for BE-8. The --be8 option must be passed
  to the final linking stage to convert the instructions to
  little-endian
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Mathieu Desnoyers pointed out that the ARM barriers were lacking:

- cmpxchg, xchg and atomic add return need memory barriers on
  architectures which can reorder the relative order in which memory
  read/writes can be seen between CPUs, which seems to include recent
  ARM architectures. Those barriers are currently missing on ARM.

- test_and_xxx_bit were missing SMP barriers.

So put these barriers in.  Provide separate atomic_add/atomic_sub
operations which do not require barriers.
Reported-Reviewed-and-Acked-by: NMathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This is needed to allow or stop the unwinding at certain points in the
kernel like exception entries.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Aaro says:
> With spinlock debugs enabled I get might_sleep() warnings when using
> ptrace.

tracked down to a missing enable_irq before calling do_undefinstr().
Reported-by: NAaro Koskinen <aaro.koskinen@nokia.com>
Tested-by: NAaro Koskinen <aaro.koskinen@nokia.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This declaration specifies the "function" type and size for various
assembly functions, mainly needed for generating the correct branch
instructions in Thumb-2.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This just leaves include/asm-arm/plat-* to deal with.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Luc Van Oostenryck reported:

  The code removed by this patch tested the irq_cpustat_t members
  __local_irq_count and __local_bh_count but these fields have
  been removed some time ago:

  http://git.kernel.org/?p=linux/kernel/git/tglx/history.git;a=commitdiff;h=3ab146c93e039dec99fec8d441a8dd046fe510cc

Fix this oversight.
Acked-by: NBill Gatliff <bgat@billgatliff.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch implements Thumb-2 application support in Linux. Original
implementation by Paul Brook with fixes for VFP and Neon by Catalin
Marinas.
Signed-off-by: NPaul Brook <paul@codesourcery.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

This patch adds a prefetch abort handler similar to the data abort one
and renames the latter for consistency. Initial implementation by Paul
Brook with some renaming by Catalin Marinas.
Signed-off-by: NPaul Brook <paul@codesourcery.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Signed-off-by: NNicolas Pitre <nico@marvell.com>

If kprobes installs a breakpoint on a "stmdb sp!, {...}" instruction,
and then single-step it by simulation from the exception context, it will
corrupt the saved regs on the stack from the previous context.

To avoid this, let's add an optional parameter to the svc_entry macro
allowing for a hole to be created on the stack before saving the
interrupted context, and use it in the undef_svc handler when kprobes
is enabled.
Signed-off-by: NNicolas Pitre <nico@marvell.com>

Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch enables the use of the Advanced SIMD (NEON) extension on
ARMv7. The NEON technology is a 64/128-bit hybrid SIMD architecture
for accelerating the performance of multimedia and signal processing
applications. The extension shares the registers with the VFP unit and
enabling/disabling and saving/restoring follow the same rules. In
addition, there are instructions that do not have the appropriate CP
number encoded, the checks being made in the call_fpe function.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The ldrt fixup code expects r9 to be set.
Signed-off-by: NNicolas Pitre <nico@marvell.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The ARM __kuser_cmpxchg routine is meant to implement an atomic cmpxchg
in user space.  It however can produce spurious false negative if a
processor exception occurs in the middle of the operation.  Normally
this is not a problem since cmpxchg is typically called in a loop until
it succeeds to implement an atomic increment for example.

Some use cases which don't involve a loop require that the operation be
100% reliable though.  This patch changes the implementation so to
reattempt the operation after an exception has occurred in the critical
section rather than abort it.

Here's a simple program to test the fix (don't use CONFIG_NO_HZ in your
kernel as this depends on a sufficiently high interrupt rate):

	#include <stdio.h>

	typedef int (__kernel_cmpxchg_t)(int oldval, int newval, int *ptr);
	#define __kernel_cmpxchg (*(__kernel_cmpxchg_t *)0xffff0fc0)

	int main()
	{
		int i, x = 0;
		for (i = 0; i < 100000000; i++) {
			int v = x;
			if (__kernel_cmpxchg(v, v+1, &x))
				printf("failed at %d: %d vs %d\n", i, v, x);
		}
		printf("done with %d vs %d\n", i, x);
		return 0;
	}
Signed-off-by: NNicolas Pitre <nico@marvell.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

get_irqnr_preamble allows machines to take some action before entering the
get_irqnr_and_base loop.  On iop we enable cp6 access.

arch_ret_to_user is added to the userspace return path to allow individual
architectures to take actions, like disabling coprocessor access, before
the final return to userspace.

Per Nicolas Pitre's note, there is no need to cp_wait on the return to user
as the latency to return is sufficient.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

bad_mode() currently prints the mode which caused the exception, and
then causes an oops dump to be printed which again displays this
information (since the CPSR in the struct pt_regs is correct.)  This
leads to processor_modes[] being shared between traps.c and process.c
with a local declaration of it.

We can clean this up by moving processor_modes[] to process.c and
removing the duplication, resulting in processor_modes[] becoming
static.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

If the kernel attempts to execute a CP1 or CP2 instruction and it
aborts, and a FP emulator is not loaded, we try to return as if to
a user context, instead of the proper kernel context.  Since the
fault came from kernel mode, we must use the kernel return paths.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

XScale cores either have a DSP coprocessor (which contains a single
40 bit accumulator register), or an iWMMXt coprocessor (which contains
eight 64 bit registers.)

Because of the small amount of state in the DSP coprocessor, access to
the DSP coprocessor (CP0) is always enabled, and DSP context switching
is done unconditionally on every task switch.  Access to the iWMMXt
coprocessor (CP0/CP1) is enabled only when an iWMMXt instruction is
first issued, and iWMMXt context switching is done lazily.

CONFIG_IWMMXT is supposed to mean 'the cpu we will be running on will
have iWMMXt support', but boards are supposed to select this config
symbol by hand, and at least one pxa27x board doesn't get this right,
so on that board, proc-xscale.S will incorrectly assume that we have a
DSP coprocessor, enable CP0 on boot, and we will then only save the
first iWMMXt register (wR0) on context switches, which is Bad.

This patch redefines CONFIG_IWMMXT as 'the cpu we will be running on
might have iWMMXt support, and we will enable iWMMXt context switching
if it does.'  This means that with this patch, running a CONFIG_IWMMXT=n
kernel on an iWMMXt-capable CPU will no longer potentially corrupt iWMMXt
state over context switches, and running a CONFIG_IWMMXT=y kernel on a
non-iWMMXt capable CPU will still do DSP context save/restore.

These changes should make iWMMXt work on PXA3xx, and as a side effect,
enable proper acc0 save/restore on non-iWMMXt capable xsc3 cores such
as IOP13xx and IXP23xx (which will not have CONFIG_CPU_XSCALE defined),
as well as setting and using HWCAP_IWMMXT properly.
Signed-off-by: NLennert Buytenhek <buytenh@wantstofly.org>
Acked-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This adds support for irqtrace for lockdep on ARM.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Patch from Nicolas Pitre

The userspace helpers in clean/arch/arm/kernel/entry-armv.S are called
directly in/from userspace. They need to cope with being called from
Thumb code.

Patch below uses the bx interworking instruction when
CONFIG_ARM_THUMB=y.

Based on an earlier patch from Paul Brook <paul@codesourcery.com>
Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Patch from Lennert Buytenhek

This patch makes the iWMMXt context switch hook use the generic
thread notifier infrastructure that was recently merged in commit
d6551e88.
Signed-off-by: NLennert Buytenhek <buytenh@wantstofly.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NJörn Engel <joern@wohnheim.fh-wedel.de>
Signed-off-by: NAdrian Bunk <bunk@stusta.de>

Patch from Lennert Buytenhek

Add the necessary kernel bits for crunch task switching.
Signed-off-by: NLennert Buytenhek <buytenh@wantstofly.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Some machine classes need to allow VFP support to be built into the
kernel, but still allow the kernel to run even though VFP isn't
present.  Unfortunately, the kernel hard-codes VFP instructions
into the thread switch, which prevents this being run-time selectable.

Solve this by introducing a notifier which things such as VFP can
hook into to be informed of events which affect the VFP subsystem
(eg, creation and destruction of threads, switches between threads.)
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Patch from Paul Brook

The example code in the source documentation for __kernel_dmb
clobbers r0 but doesn't list it the asm clobber list.
Signed-off-by: NPaul Brook <paul@codesourcery.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Allow the individual coprocessor handlers to decide when to enable
interrupts, rather than unconditionally enabling them.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

CONFIG_CPU_MPCORE has never been a configuration symbol - it should
be CONFIG_CPU_32v6K.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

 negative

Patch from Nicolas Pitre

The pre ARMv5 implementation can be aborted if an exception occurs in
the middle of it.  Because of that, the ARMv6 implementation doesn't
re-attempt the operation on a failed strex either.  Let's make this
transient nature of such a false positive more explicit in the
definition.
Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Patch from Nicolas Pitre

The cmpxchg emulation on pre-ARMv5 relies on user code executed from a
kernel address.  If the operation cannot complete atomically, it is
aborted from the usr_entry macro by clearing the Z flag.  This clearing
of the Z flag is done whenever the user pc is above TASK_SIZE.

However this "pc >= TASK_SIZE" test cannot work in the non MMU case.
Worse: the current code will corrupt the Z flag on every entry to the
kernel.

Let's disable it in the non MMU case for now.  Using NPTL on non MMU
targets needs to be worked out anyway.
Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Patch from Nicolas Pitre

This is kernel provided user space code.

Since a syscall is used, it has to be updated to work with EABI.
Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Patch from Nicolas Pitre

The ARM EABI says that the stack pointer has to be 64-bit aligned for
reasons already mentioned in patch #3101 when calling C functions.

We therefore must verify and adjust sp accordingly when taking an
exception from kernel mode since sp might not necessarily be 64-bit
aligned if the exception occurs in the middle of a kernel function.

If the exception occurs while in user mode then no sp fixup is needed as
long as sizeof(struct pt_regs) as well as any additional syscall data
stack space remain multiples of 8.
Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch adds register switch support in nommu mode.
Signed-off-by: NHyok S. Choi <hyok.choi@samsung.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

arch/arm/kernel/entry-armv.S has contained a comment suggesting
that asm/hardware.h and asm/arch/irqs.h should be moved into the
asm/arch/entry-macro.S include.  So move the includes to these
two files as required.

Add missing includes (asm/hardware.h, asm/io.h) to asm/arch/system.h
includes which use those facilities, and remove asm/io.h from
kernel/process.c.

Remove other unnecessary includes from arch/arm/kernel, arch/arm/mm
and arch/arm/mach-footbridge.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Patch from Nicolas Pitre

Strictly speaking, the NPTL kernel helpers are required for pre ARMv6
only.  They are available on ARMv6+ as well for obvious compatibility
reasons.  However there are cases where extra memory barriers are needed
when using an SMP ARMv6 machine but not on pre-ARMv6.

This patch adds a memory barrier kernel helper that glibc can use as
needed for pre-ARMv6 binaries to be forward compatible with an SMP
kernel on ARMv6, as well as the necessary dmb instructions to the
cmpxchg helper.
Signed-off-by: NNicolas Pitre <nico@cam.org>
Acked-by: NDaniel Jacobowitz <dan@codesourcery.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add infrastructure for supporting per-cpu local timers to update
the profiling information and update system time accounting.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Patch from Nicolas Pitre

Since we know the value of cpsr on entry, we can replace the bic+orr with
a single eor.  Also remove a possible result delay (at least on XScale).
Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>