This allows the cache/processor/fault glue to be more easily used
from assembler code.  Tested on Assabet and Tegra 2.
Tested-by: NColin Cross <ccross@android.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The v6 cache call optimization was disabled to allow the optional block
cache operations to be subsituted on CPUs which supported those
operations.  However, as that functionality was removed, we no longer
need to prevent this optimization being taken advantage of.

The v7 cache call optimization was just a copy of the v6, so also fix
that too.
Tested-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Introduce a CPU_V6K configuration option for platforms to select if they
have a V6K CPU core.  This allows us to identify whether we need to
support ARMv6 CPUs without the V6K SMP extensions at build time.

Currently CPU_V6K is just an alias for CPU_V6, and all places which
reference CPU_V6 are replaced by (CPU_V6 || CPU_V6K).

Select CPU_V6K from platforms which are known to be V6K-only.
Acked-by: NTony Lindgren <tony@atomide.com>
Tested-by: NSourav Poddar <sourav.poddar@ti.com>
Tested-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Do this by adding flush_icache_all to cache_fns for ARMv6 and 7.
As flush_icache_all may neeed to be called from flush_kern_cache_all,
add it as the first entry in the cache_fns.

Note that now we can remove the ARM_ERRATA_411920 dependency
to !SMP so it can be selected on UP ARMv6 processors, such
as omap2.
Signed-off-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NAnand Gadiyar <gadiyar@ti.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Since page cache pages are now considered 'dirty' by default, the cache
flushing is handled via __flush_dcache_page() when a page gets mapped to
user space. Highmem pages on VIVT systems are flushed during kunmap()
and flush_kernel_dcache_page() was already a no-op in this case.

ARCH_HAS_FLUSH_KERNEL_DCACHE_PAGE is still defined since ARM needs
specific implementations for flush_kernel_vmap_range() and
invalidate_kernel_vmap_range().

Cc: Nicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

There are places in Linux where writes to newly allocated page cache
pages happen without a subsequent call to flush_dcache_page() (several
PIO drivers including USB HCD). This patch changes the meaning of
PG_arch_1 to be PG_dcache_clean and always flush the D-cache for a newly
mapped page in update_mmu_cache().

The patch also sets the PG_arch_1 bit in the DMA cache maintenance
function to avoid additional cache flushing in update_mmu_cache().
Tested-by: NRabin Vincent <rabin.vincent@stericsson.com>
Cc: Nicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The standard I-cache Invalidate All (ICIALLU) and Branch Predication
Invalidate All (BPIALL) operations are not automatically broadcast to
the other CPUs in an ARMv7 MP system. The patch adds the Inner Shareable
variants, ICIALLUIS and BPIALLIS, if ARMv7 and SMP.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

To avoid #include collisions with subsequent patches in the series, this
patch moves the outer_cache definitions to a separate asm/outercache.h
file.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch fix the below build error for arm1026ej-s processor (IntegratorCP/arm1026ej-s board).
  CC      init/main.o
In file included from include/linux/highmem.h:8,
                 from include/linux/pagemap.h:10,
                 from include/linux/mempolicy.h:62,
                 from init/main.c:52:
arch/arm/include/asm/cacheflush.h:134:2: error: #error Unknown cache maintainence model
make[1]: *** [init/main.o] Erreur 1
make: *** [init] Erreur 2
Signed-off-by: NAbdoulaye Walsimou Gaye <walsimou@walsimou.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

These are now unused, and so can be removed.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-By: NSantosh Shilimkar <santosh.shilimkar@ti.com>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-By: NSantosh Shilimkar <santosh.shilimkar@ti.com>

ARM cannot prevent cache movein, so this patch implements both the
flush and invalidate pieces of the API.
Signed-off-by: NJames Bottomley <James.Bottomley@suse.de>

The comments in cacheflush.h should follow what's in
struct cpu_cache_fns. The comments for V6 and V7 are
unnecessary.
Signed-off-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

... and rename the function since it no longer operates on just
pages.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

There is not enough users to warrant its existence, and it is actually
an obstacle to progress with the new DMA API which cannot cover this
case properly.

To keep backward compatibility, let's perform the necessary custom
cache maintenance locally in the only driver affected.
Signed-off-by: NNicolas Pitre <nico@marvell.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We had two copies of the wrapper code for VIVT cache flushing - one in
asm/cacheflush.h and one in arch/arm/mm/flush.c.  Reduce this down to
one common copy.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Mtdblock driver doesn't call flush_dcache_page for pages in request.  So,
this causes problems on architectures where the icache doesn't fill from
the dcache or with dcache aliases.  The patch fixes this.

The ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE symbol was introduced to avoid
pointless empty cache-thrashing loops on architectures for which
flush_dcache_page() is a no-op.  Every architecture was provided with this
flush pages on architectires where ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE is
equal 1 or do nothing otherwise.

See "fix mtd_blkdevs problem with caches on some architectures" discussion
on LKML for more information.
Signed-off-by: NIlya Loginov <isloginov@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Peter Horton <phorton@bitbox.co.uk>
Cc: "Ed L. Cashin" <ecashin@coraid.com>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Errata 411920 indicates that any "invalidate entire instruction cache"
operation can fail if the right conditions are present.  This is not
limited just to those operations in flush.c, but elsewhere.  Place the
workaround in the already existing __flush_icache_all() function
instead.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Makes code futureproof against the impending change to mm->cpu_vm_mask.

It's also a chance to use the new cpumask_ ops which take a pointer
(the older ones are deprecated, but there's no hurry for arch code).
Signed-off-by: NRusty Russell <rusty@rustcorp.com.au>

Without this, the default implementation is a no op which is completely
wrong with a VIVT cache, and usage of sg_copy_buffer() produces
unpredictable results.
Tested-by: NSebastian Andrzej Siewior <bigeasy@breakpoint.cc>

CC: stable@kernel.org
Signed-off-by: NNicolas Pitre <nico@marvell.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Adds support for Faraday FA526 core. This core is used at least by:
Cortina Systems Gemini and Centroid family
Cavium Networks ECONA family
Grain Media GM8120
Pixelplus ImageARM
Prolific PL-1029
Faraday IP evaluation boards

v2:
- move TLB_BTB to separate patch
- update copyrights
Signed-off-by: NPaulius Zaleckas <paulius.zaleckas@teltonika.lt>

"""The Marvell® PXA168 processor is the first in a family of application
processors targeted at mass market opportunities in computing and consumer
devices. It balances high computing and multimedia performance with low
power consumption to support extended battery life, and includes a wealth
of integrated peripherals to reduce overall BOM cost .... """

See http://www.marvell.com/featured/pxa168.jsp for more information.

  1. Marvell Mohawk core is a hybrid of xscale3 and its own ARM core,
     there are many enhancements like instructions for flushing the
     whole D-cache, and so on

  2. Clock reuses Russell's common clkdev, and added the basic support
     for UART1/2.

  3. Devices are a bit different from the 'mach-pxa' way, the platform
     devices are now dynamically allocated only when necessary (i.e.
     when pxa_register_device() is called). Description for each device
     are stored in an array of 'struct pxa_device_desc'. Now that:

     a. this array of device description is marked with __initdata and
        can be freed up system is fully up

     b. which means board code has to add all needed devices early in
        his initializing function

     c. platform specific data can now be marked as __initdata since
        they are allocated and copied by platform_device_add_data()

  4. only the basic UART1/2/3 are added, more devices will come later.
Signed-off-by: NJason Chagas <chagas@marvell.com>
Signed-off-by: NEric Miao <eric.miao@marvell.com>

... and fix those drivers that were incorrectly relying upon
that include.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

In case of non-aliasing VIPT caches, there is no need to flush the whole
cache when new mapping is created. The patch introduces this condition
check. In the non-aliasing VIPT case flush_cache_vmap() needs a DSB
since the set_pte_at() function called from vmap_pte_range() does not
have such barrier (done usually via TLB flushing functions).
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Rather than pollute asm/cacheflush.h with the cache type definitions,
move them to asm/cachetype.h, and include this new header where
necessary.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Move platform independent header files to arch/arm/include/asm, leaving
those in asm/arch* and asm/plat* alone.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

cacheflush.h was doing:

... VIVT only stuff
... VIPT only stuff
... VIVT or VIPT stuff

which is clearly bogus - we would only ever use the "VIVT or VIPT" case
when both VIVT and VIPT are not selected.  Fix this.

Add comments to each case, including noting the impossibility of
correctly detecting the cache type of ARM926 and ARMv6 cores from
the cache type register in the "VIVT or VIPT" case.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

mapping->tree_lock has no read lockers.  convert the lock from an rwlock
to a spinlock.
Signed-off-by: NNick Piggin <npiggin@suse.de>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Hugh Dickins <hugh@veritas.com>
Cc: "Paul E. McKenney" <paulmck@us.ibm.com>
Reviewed-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This patch adds the I-cache invalidation in update_mmu_cache if the
corresponding vma is marked as executable. It also invalidates the
I-cache if a thread migrates to a CPU it never ran on.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch adds support for the L1 D cache range operations that
are supported by the Marvell Discovery Duo and Marvell Kirkwood
ARM SoCs.
Signed-off-by: NStanislav Samsonov <samsonov@marvell.com>
Acked-by: NSaeed Bishara <saeed@marvell.com>
Reviewed-by: NNicolas Pitre <nico@marvell.com>
Signed-off-by: NLennert Buytenhek <buytenh@marvell.com>

The Feroceon is a family of independent ARMv5TE compliant CPU core
implementations, supporting a variable depth pipeline and out-of-order
execution.  The Feroceon is configurable with VFP support, and the
later models in the series are superscalar with up to two instructions
per clock cycle.

This patch adds the initial low-level cache/TLB handling for this core.
Signed-off-by: NAssaf Hoffman <hoffman@marvell.com>
Reviewed-by: NTzachi Perelstein <tzachi@marvell.com>
Reviewed-by: NNicolas Pitre <nico@marvell.com>
Reviewed-by: NLennert Buytenhek <buytenh@marvell.com>
Acked-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This fixes a regression from around 2.6.18, consistent_sync() will now BUG()
under these circumstances.  The use of consistent_sync() was a hack, replacing
it's usage here with a new function, flush_ioremap_region().
Signed-off-by: NJared Hulbert <jaredeh@gmail.com>
Acked-by: NPavel Pisa <pisa@cmp.felk.cvut.cz>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ARMv7 can have VIPT, PIPT or ASID-tagged VIVT I-cache. This patch
adds the necessary invalidation of the I-cache when the ASID numbers
are re-used.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch renames the old __cacheid_* macros to __cacheid_*_prev7 and adds
support for the new format.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch adds support for the ARMv7 cores.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The DMA cache handling functions take virtual addresses, but in the
form of unsigned long arguments.  This leads to a little confusion
about what exactly they take.  So, convert them to take const void *
instead.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The outer cache can be L2 as on RealView/EB MPCore platform or even L3
or further on ARMv7 cores. This patch adds the generic support for
flushing the outer cache in the DMA operations.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

fuse does not work on ARM due to cache incoherency issues - fuse wants
to use get_user_pages() to copy data from the current process into
kernel space.  However, since this accesses userspace via the kernel
mapping, the kernel mapping can be out of date wrt data written to
userspace.

This can lead to unpredictable behaviour (in the case of fuse) or data
corruption for direct-IO.

This resolves debian bug #402876
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>