In preparation for removing volatile from the atomic_t definition, this
patch adds a volatile cast to all the atomic read functions.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

In preparation for perf-events support, ARM needs to support atomic64_t
operations. v6k and above support the ldrexd and strexd instructions to
do just that.

This patch adds atomic64 support to the ARM architecture. v6k and above
make use of new instructions whilst older cores fall back on the generic
solution using spinlocks. If and when v7-M cores are supported by Linux,
they will need to fall back on the spinlock implementation too.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The patch adds a CLREX or dummy STREX to the exception return path. This
is needed because several atomic/locking operations use a pair of
LDREX/STREXEQ and the EQ condition may not always be satisfied. This
would leave the exclusive monitor status set and may cause problems with
atomic/locking operations in the interrupted code.

With this patch, the atomic_set() operation can be a simple STR
instruction (on SMP systems, the global exclusive monitor is cleared by
STR anyway). Clearing the exclusive monitor during context switch is no
longer needed as this is handled by the exception return path anyway.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Reported-by: NJamie Lokier <jamie@shareable.org>

fix the following 'make includecheck' warning:

  arch/arm/include/asm/atomic.h: asm/system.h is included more than once.
Signed-off-by: NJaswinder Singh Rajput <jaswinderrajput@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The existing asm-generic/atomic.h only defines the
atomic_long type. This renames it to atomic-long.h
so we have a place to add a truly generic atomic.h
that can be used on all non-SMP systems.
Signed-off-by: NRemis Lima Baima <remis.developer@googlemail.com>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NIngo Molnar <mingo@elte.hu>

Mathieu Desnoyers pointed out that the ARM barriers were lacking:

- cmpxchg, xchg and atomic add return need memory barriers on
  architectures which can reorder the relative order in which memory
  read/writes can be seen between CPUs, which seems to include recent
  ARM architectures. Those barriers are currently missing on ARM.

- test_and_xxx_bit were missing SMP barriers.

So put these barriers in.  Provide separate atomic_add/atomic_sub
operations which do not require barriers.
Reported-Reviewed-and-Acked-by: NMathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The atomic_t type cannot currently be used in some header files because it
would create an include loop with asm/atomic.h.  Move the type definition
to linux/types.h to break the loop.
Signed-off-by: NMatthew Wilcox <willy@linux.intel.com>
Cc: Huang Ying <ying.huang@intel.com>
Cc: <linux-arch@vger.kernel.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Move platform independent header files to arch/arm/include/asm, leaving
those in asm/arch* and asm/plat* alone.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

atomic_add_unless as inline. Remove system.h atomic.h circular dependency.
I agree (with Andi Kleen) this typeof is not needed and more error
prone. All the original atomic.h code that uses cmpxchg (which includes
the atomic_add_unless) uses defines instead of inline functions,
probably to circumvent a circular dependency between system.h and
atomic.h on powerpc (which my patch addresses). Therefore, it makes
sense to use inline functions that will provide type checking.

atomic_add_unless as inline. Remove system.h atomic.h circular dependency.
Digging into the FRV architecture shows me that it is also affected by
such a circular dependency. Here is the diff applying this against the
rest of my atomic.h patches.

It applies over the atomic.h standardization patches.
Signed-off-by: NMathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Trying to build some code using atomic_clear_mask() on a ARM v6
processor with a recent compiler (tried with gcc version 4.1.1
(CodeSourcery ARM Sourcery G++ 2006q3-26), but
all gcc > 4.1 might be affected) results in the following:

/tmp/ccWKLJV8.s: Assembler messages:
/tmp/ccWKLJV8.s:581: Error: instruction does not accept this addressing
mode -- `ldrex r0,r3'
/tmp/ccWKLJV8.s:583: Error: instruction does not accept this addressing
mode -- `strex r1,r0,r3'

Older gcc (like gcc version 4.0.0 (DENX ELDK 4.1 4.0.0)) have no problem
with this.

The patch below fixes the compile error. I also verified that gcc-4.0.0 generates identical code using both forms.
Signed-off-by: NStelian Pop <stelian@popies.net>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Now that we have raw_* variants of local_irq_$FOO(), switch the atomic
helpers over to use those raw_* variants.  This is necessary when using
lockdep on pre-ARMv6 hardware, as lockdep uses atomic_t counters in the
trace_hardirqs_off() path.
Signed-off-by: NLennert Buytenhek <buytenh@wantstofly.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NDavid Woodhouse <dwmw2@infradead.org>

add atomic_xchg() to all the architectures. Needed by the new mutex code.
Signed-off-by: NIngo Molnar <mingo@elte.hu>
Signed-off-by: NArjan van de Ven <arjan@infradead.org>

Several counters already have the need to use 64 atomic variables on 64 bit
platforms (see mm_counter_t in sched.h).  We have to do ugly ifdefs to fall
back to 32 bit atomic on 32 bit platforms.

The VM statistics patch that I am working on will also make more extensive
use of atomic64.

This patch introduces a new type atomic_long_t by providing definitions in
asm-generic/atomic.h that works similar to the c "long" type.  Its 32 bits
on 32 bit platforms and 64 bits on 64 bit platforms.

Also cleans up the determination of the mm_counter_t in sched.h.
Signed-off-by: NChristoph Lameter <clameter@sgi.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Since atomic.h does not include types.h, u32 may not be defined.
Since atomics are supposed to work on unsigned long quantities,
use unsigned long instead.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

atomic.h, bitops.h and mmu_context.h are using likely/unlikely.
thread_info.h uses __attribute_const__.  Hence these files require
linux/compiler.h to be included.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Patch from Nicolas Pitre

If 'old' and 'oldval' are different then 'res' never gets set.  In that
case, if ever %0 happened to contain anything but zero (rather likely)
then the code will loop forever (or until another CPU just come along
and change the atomic value to match 'old' which is rather unlikely).
Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Introduce an atomic_inc_not_zero operation.  Make this a special case of
atomic_add_unless because lockless pagecache actually wants
atomic_inc_not_negativeone due to its offset refcount.
Signed-off-by: NNick Piggin <npiggin@suse.de>
Cc: "Paul E. McKenney" <paulmck@us.ibm.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Introduce an atomic_cmpxchg operation.
Signed-off-by: NNick Piggin <npiggin@suse.de>
Cc: "Paul E. McKenney" <paulmck@us.ibm.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!