commit 522d7dec(futex: Remove redundant pagefault_disable in
futex_atomic_cmpxchg_inatomic()) added a bogus comment.

/* Note that preemption is disabled by futex_atomic_cmpxchg_inatomic
 * call sites. */

Bogus in two aspects:

1) pagefault_disable != preempt_disable even if the mechanism we use
   is the same

2) we have a call site which deliberately does not disable pagefaults
   as it wants the possible fault to be handled - though that has been
   changed for consistency reasons now.

Sigh. I really should have seen that when committing the above. :(
Catched-by-and-rightfully-ranted-at-by: NLinus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
LKML-Reference: <alpine.LFD.2.00.1103141126590.2787@localhost6.localdomain6>
Cc: Michel Lespinasse <walken@google.com>
Cc: Darren Hart <darren@dvhart.com>

Change futex_atomic_op_inuser and futex_atomic_cmpxchg_inatomic
prototypes to use u32 types for the futex as this is the data type the
futex core code uses all over the place.
Signed-off-by: NMichel Lespinasse <walken@google.com>
Cc: Darren Hart <darren@dvhart.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Matt Turner <mattst88@gmail.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: David Howells <dhowells@redhat.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: "James E.J. Bottomley" <jejb@parisc-linux.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Chris Metcalf <cmetcalf@tilera.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
LKML-Reference: <20110311025058.GD26122@google.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>

The cmpxchg_futex_value_locked API was funny in that it returned either
the original, user-exposed futex value OR an error code such as -EFAULT.
This was confusing at best, and could be a source of livelocks in places
that retry the cmpxchg_futex_value_locked after trying to fix the issue
by running fault_in_user_writeable().
    
This change makes the cmpxchg_futex_value_locked API more similar to the
get_futex_value_locked one, returning an error code and updating the
original value through a reference argument.
Signed-off-by: NMichel Lespinasse <walken@google.com>
Acked-by: Chris Metcalf <cmetcalf@tilera.com>  [tile]
Acked-by: Tony Luck <tony.luck@intel.com>  [ia64]
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Tested-by: Michal Simek <monstr@monstr.eu>  [microblaze]
Acked-by: David Howells <dhowells@redhat.com> [frv]
Cc: Darren Hart <darren@dvhart.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Matt Turner <mattst88@gmail.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: "James E.J. Bottomley" <jejb@parisc-linux.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
LKML-Reference: <20110311024851.GC26122@google.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>

kernel/futex.c disables page faults before calling
futex_atomic_cmpxchg_inatomic(), so there is no need to do it again
within that function.
Signed-off-by: NMichel Lespinasse <walken@google.com>
Cc: Darren Hart <darren@dvhart.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Matt Turner <mattst88@gmail.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: David Howells <dhowells@redhat.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: "James E.J. Bottomley" <jejb@parisc-linux.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Chris Metcalf <cmetcalf@tilera.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
LKML-Reference: <20110311024731.GB26122@google.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>

Since commit 6fc31d54 this comment is no longer true.
Signed-off-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Commit

	06824ba8 (ARM: tlb: delay page freeing for SMP and ARMv7 CPUs)

introduced a build failure for builds with CONFIG_SWAP=n:

	In file included from arch/arm/mm/init.c:27:
	arch/arm/include/asm/tlb.h: In function 'tlb_flush_mmu':
	arch/arm/include/asm/tlb.h:101: error: implicit declaration of function 'release_pages'
	arch/arm/include/asm/tlb.h: In function 'tlb_remove_page':
	arch/arm/include/asm/tlb.h:165: error: implicit declaration of function 'page_cache_release'

as linux/swap.h doesn't include linux/pagemap.h but actually needs it
(see comments in linux/swap.h as to why this is.)

Fix that by #including <linux/pagemap.h> in <asm/pgalloc.h> as it's done
by x86.
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Acked-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

There's no need to noMMU to put tlb_flush() in asm/tlbflush.h - it's
part of the tlb shootdown interface.  Move it to asm/tlb.h instead, as
per x86.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We need to delay freeing any mapped page on SMP and ARMv7 systems to
ensure that the data is not accessed by other CPUs, or is used for
speculative prefetch with ARMv7.  This includes not only mapped pages
but also pages used for the page tables themselves.

This avoids races with the MMU/other CPUs accessing pages after they've
been freed but before we've invalidated the TLB.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

In sysctl_soft_reset(), switch to slow mode before resetting the system
via the system controller.  This is required.
Reviewed-by: NStanley Miao <stanley.miao@windriver.com>
Signed-off-by: NShiraz Hashim <shiraz.hashim@st.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The effect of cache sync operation is to drain the store buffer and
wait for all internal buffers to be empty. In normal conditions, store
buffer is able to merge the normal memory writes within its 32-byte
data buffers.  Due to this erratum present in r3p0, the effect of cache
sync operation on the store buffer still remains when the operation
completes. This means that the store buffer is always asked to drain
and this prevents it from merging any further writes.

This can severely affect performance on the write traffic esp. on
Normal memory NC one.

The proposed workaround is to replace the normal offset of cache sync
operation(0x730) by another offset targeting an unmapped PL310
register 0x740.
Signed-off-by: Nsrinidhi kasagar <srinidhi.kasagar@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Ensure that the ISA/PCI IO space accessors are properly ordered on
ARMv6+ architectures.  These should always be ordered with respect to
all other accesses.

This also fixes __iormb() and __iowmb() not being visible to ioread/
iowrite if a platform defines its own MMIO accessors.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Changing the virt_to_phys() argument to "const volatile void *" avoids
compiler warnings in some situations where this function is used.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NStephen Boyd <sboyd@codeaurora.org>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Timers on Versatile Express mainboard are used as system clock/event
sources. Driver assumes that they are clocked with 1MHz signal.
Old V2M firmware apparently configured it by default, but on newer
boards one can observe that "sleep 1" command takes over 30 seconds
to finish, as the timers are fed with 32kHz instead...

This patch performs required magic and also removes code clearing
timer's control registers, as exactly the same operations are
performed by the timer driver few jiffies later.
Signed-off-by: NPawel Moll <pawel.moll@arm.com>
Tested-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The CLZ instruction does not alter the condition flags, so remove the
"cc" clobber from the inline asm for fls().
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRabin Vincent <rabin@rab.in>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

sched_clock is supposed to be initialized early - in the recently added
init_early platform hook.  However, in doing so we end up calling
mod_timer() before the timer lists are initialized, resulting in an
oops.

Split the initialization in two - the part which the platform calls
early which starts things off.  The addition of the timer can be
delayed until after we have more of the kernel initialized - when the
normal time sources are initialized.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add ARM support for the DMA debug infrastructure, which allows the
DMA API usage to be debugged.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The commit 6ac6b817 (ARM: pxa: encode
IRQ number into .nr_irqs) removed definition of ITE_LAST_IRQ which
caused the following build error:

CC      arch/arm/common/it8152.o
arch/arm/common/it8152.c: In function 'it8152_init_irq':
arch/arm/common/it8152.c:86: error: 'IT8152_LAST_IRQ' undeclared (first use in this function)
arch/arm/common/it8152.c:86: error: (Each undeclared identifier is reported only once
arch/arm/common/it8152.c:86: error: for each function it appears in.)
make[2]: *** [arch/arm/common/it8152.o] Error 1

Defining the IT8152_LAST_IRQ in the arch/arm/include/hardware/it8152.c
fixes the build.
Signed-off-by: NMike Rapoport <mike@compulab.co.il>
Signed-off-by: NEric Miao <eric.y.miao@gmail.com>

Replace the page_to_dma() and dma_to_page() macros with their PFN
equivalents.  This allows us to map parts of memory which do not have
a struct page allocated to them to bus addresses.  This will be used
internally by dma_alloc_coherent()/dma_alloc_writecombine().

Build tested on Versatile, OMAP1, IOP13xx and KS8695.
Tested-by: NJanusz Krzysztofik <jkrzyszt@tis.icnet.pl>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch fixes below build error by adding the missing asm/memory.h,
which is needed for arch_is_coherent().

$ make pxa3xx_defconfig; make
  CC      init/do_mounts_rd.o
In file included from include/linux/list_bl.h:5,
                 from include/linux/rculist_bl.h:7,
                 from include/linux/dcache.h:7,
                 from include/linux/fs.h:381,
                 from init/do_mounts_rd.c:3:
include/linux/bit_spinlock.h: In function 'bit_spin_unlock':
include/linux/bit_spinlock.h:61: error: implicit declaration of function 'arch_is_coherent'
make[1]: *** [init/do_mounts_rd.o] Error 1
make: *** [init] Error 2
Signed-off-by: NAxel Lin <axel.lin@gmail.com>
Acked-by: NPeter Huewe <peterhuewe@gmx.de>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

sev is used to send wakeup event to other cores in ARMv6K and above.
This has been moved from platform specific part to standard common
ARM header file (asm/system.h). Also introduced wfi() and wfe().
Signed-off-by: NShiraz Hashim <shiraz.hashim@st.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This allows platforms to hook into the initialization early to setup
things like scheduler clocks, etc.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Rather than storing each machine init hook separately, store a
pointer to the machine description record and dereference this
instead.  This pointer is only available while the init sections
are present, which is not a problem as we only use it from init
code.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Per subarch interrupt handler macros V3.

This patch breaks out code from the irq_handler macro
into arch_irq_handler and arch_irq_handler_default.

The macros are put in the header file "entry-macro-multi.S"

The arch_irq_handler_default macro is designed to be
used by irq_handler in entry-armv.S while arch_irq_handler
is suitable for per-subarch use.
Signed-off-by: NMagnus Damm <damm@opensource.se>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Normally different ARM platform has different way to decode the IRQ
hardware status and demultiplex to the corresponding IRQ handler.
This is highly optimized by macro irq_handler in entry-armv.S, and
each machine defines their own macro to decode the IRQ number.
However, this prevents multiple machine classes to be built into a
single kernel.

By allowing each machine to specify thier own handler, and making
function pointer 'handle_arch_irq' to point to it at run time, this
can be solved. And introduce CONFIG_MULTI_IRQ_HANDLER to allow both
solutions to work.

Comparing with the highly optimized macro of irq_handler, the new
function must be written with care not to lose too much performance.
And the IPI stuff on SMP is expected to move to the provided arch
IRQ handler as well.

The assembly code to invoke handle_arch_irq is optimized by Russell
King.
Signed-off-by: NEric Miao <eric.miao@canonical.com>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

... and also remove misleading comment stating that this header is
auto-generated.
Signed-off-by: NStephen Warren <swarren@nvidia.com>
Acked-by: NUwe Kleine-Knig <u.kleine-koenig@pengutronix.de>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Provide common sched_clock() infrastructure for platforms to use to
create a 64-bit ns based sched_clock() implementation from a counter
running at a non-variable clock rate.

This implementation is based upon maintaining an epoch for the counter
and an epoch for the nanosecond time.  When we desire a sched_clock()
time, we calculate the number of counter ticks since the last epoch
update, convert this to nanoseconds and add to the epoch nanoseconds.

We regularly refresh these epochs within the counter wrap interval.
We perform a similar calculation as above, and store the new epochs.

We read and write the epochs in such a way that sched_clock() can easily
(and locklessly) detect when an update is in progress, and repeat the
loading of these constants when they're known not to be stable.  The
one caveat is that sched_clock() is not called in the middle of an
update.  We achieve that by disabling IRQs.

Finally, if the clock rate is known at compile time, the counter to ns
conversion factors can be specified, allowing sched_clock() to be tightly
optimized.  We ensure that these factors are correct by providing an
initialization function which performs a run-time check.
Acked-by: NPeter Zijlstra <peterz@infradead.org>
Tested-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: NWill Deacon <will.deacon@arm.com>
Tested-by: NMikael Pettersson <mikpe@it.uu.se>
Tested-by: NEric Miao <eric.y.miao@gmail.com>
Tested-by: NOlof Johansson <olof@lixom.net>
Tested-by: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The hardware page tables use an XN bit 'execute never'.  Historically,
we've had a Linux 'execute allow' bit, in the positive sense.  Get rid
of this artifact as future hardware will continue to have the XN sense.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

FIRST_USER_PGD_NR is now unnecessary, as this has been replaced by
FIRST_USER_ADDRESS except in the architecture code.  Fix up the last
usage of FIRST_USER_PGD_NR, and remove the definition.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We have two places where we create identity mappings - one when we bring
secondary CPUs online, and one where we setup some mappings for soft-
reboot.  Combine these two into a single implementation.  Also collect
the identity mapping deletion function.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This switches the ordering of the Linux vs hardware page tables in
each page, thereby eliminating some of the arithmetic in the page
table walks.  As we now place the Linux page table at the beginning
of the page, we can deal with the offset in the pgt by simply masking
it away, along with the other control bits.

This also makes the arithmetic all be positive, rather than a mixture.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

When a CPU is hot unplugged, the generic tick code cleans up the
clock event device, but fails to call down to the device's set_mode
function to actually shut the device down.

To work around this, we've historically had a local_timer_stop()
callback out of the hotplug code.  However, this adds needless
complexity when we have the clock event device itself available.

Explicitly call the clock event device's set_mode function with
CLOCK_EVT_MODE_UNUSED, so that the hardware can be cleanly shutdown
without any special external callbacks.  When/if the generic code
is fixed, percpu_timer_stop() can be killed off.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

  * __fixup_smp_on_up has been modified with support for the
    THUMB2_KERNEL case.  For THUMB2_KERNEL only, fixups are split
    into halfwords in case of misalignment, since we can't rely on
    unaligned accesses working before turning the MMU on.

    No attempt is made to optimise the aligned case, since the
    number of fixups is typically small, and it seems best to keep
    the code as simple as possible.

  * Add a rotate in the fixup_smp code in order to support
    CPU_BIG_ENDIAN, as suggested by Nicolas Pitre.

  * Add an assembly-time sanity-check to ALT_UP() to ensure that
    the content really is the right size (4 bytes).

    (No check is done for ALT_SMP().  Possibly, this could be fixed
    by splitting the two uses ot ALT_SMP() (ALT_SMP...SMP_UP versus
    ALT_SMP...SMP_UP_B) into two macros.  In the first case,
    ALT_SMP needs to expand to >= 4 bytes, not == 4.)

  * smp_mpidr.h (which implements ALT_SMP()/ALT_UP() manually due
    to macro limitations) has not been modified: the affected
    instruction (mov) has no 16-bit encoding, so the correct
    instruction size is satisfied in this case.

  * A "mode" parameter has been added to smp_dmb:

    smp_dmb arm @ assumes 4-byte instructions (for ARM code, e.g. kuser)
    smp_dmb     @ uses W() to ensure 4-byte instructions for ALT_SMP()

    This avoids assembly failures due to use of W() inside smp_dmb,
    when assembling pure-ARM code in the vectors page.

    There might be a better way to achieve this.

  * Kconfig: make SMP_ON_UP depend on
    (!THUMB2_KERNEL || !BIG_ENDIAN) i.e., THUMB2_KERNEL is now
    supported, but only if !BIG_ENDIAN (The fixup code for Thumb-2
    currently assumes little-endian order.)

Tested using a single generic realview kernel on:
	ARM RealView PB-A8 (CONFIG_THUMB2_KERNEL={n,y})
	ARM RealView PBX-A9 (SMP)
Signed-off-by: NDave Martin <dave.martin@linaro.org>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

With "ARM: CPU hotplug: remove bug checks in platform_cpu_die()", we
now do not use hard_smp_processor_id(), we no longer need to read the
hardware processor ID.  Remove the include providing this function.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

There is a certain amount of smp_prepare_cpus() which doesn't belong
in the platform support code - that is, code which is invariant to the
SMP implementation.  Move this code into arch/arm/kernel/smp.c, and
add a platform_ prefix to the original function.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The IPI and local timer interrupts weren't being properly accounted
for in /proc/stat.  Collect them from the irq_stat structure, and
return their sum.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This separates out the individual IPI interrupt counts from the
total IPI count, which allows better visibility of what IPIs are
being used for.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

As per x86, align the initial column according to how many IRQs we
have.  Also, provide an english explaination for the 'LOC:' and
'IPI:' lines.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Move the ipi_count into irq_stat, which allows the ipi_data structure
to be entirely removed.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Provide __inc_irq_stat() and __get_irq_stat() to increment and
read the irq stat counters.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>