There is not enough users to warrant its existence, and it is actually
an obstacle to progress with the new DMA API which cannot cover this
case properly.

To keep backward compatibility, let's perform the necessary custom
cache maintenance locally in the only driver affected.
Signed-off-by: NNicolas Pitre <nico@marvell.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Dirk Behme reported instability on ARM11 SMP (VIPT non-aliasing cache)
caused by the dynamic linker changing protection on text pages to write
GOT entries.  The problem is due to an interaction between the write
faulting code providing new anonymous pages which are incoherent with
the I-cache due to write buffering, and the I-cache not having been
invalidated.

a4db94d plugs the hole with the data cache coherency.  This patch
provides the other half of the fix by flushing the I-cache in
flush_cache_range() for VM_EXEC VMAs (which is what we have when the
region is being made executable again.)  This ensures that the I-cache
will be up to date with the newly COW'd pages.

Note: if users are writing instructions, then they still need to use
the ARM sys_cacheflush API to ensure that the caches are correctly
synchronized.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

flush_cache_mm() is called in two cases:
1. when a process exits, just before the page tables are torn down.
   We can allow the stale lines to evict themselves over time without
   causing any harm.

2. when a process forks, and we've allocated a new ASID.
   The instruction cache issues are dealt with as pages are brought
   into the new process address space.  Flushing the I-cache here is
   therefore unnecessary.

However, we must keep the VIPT aliasing D-cache flush to ensure that
any dirty cache lines are not written back after the pages have been
reallocated for some other use - which would result in corruption.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The I and D caches for copy-on-write pages on processors with
write-allocate caches become incoherent causing problems on application
relying on CoW for text pages (dynamic linker relocating symbols in a
text page). This patch flushes the D-cache for such pages.

Cc: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Both call sites for __flush_dcache_page() end up calling
__flush_icache_all() themselves, so having __flush_dcache_page() do
this as well is wasteful.  Remove the duplicated icache flushing.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

If running in non-secure mode accessing
some registers of l2x0 will fault. So
check if l2x0 is already enabled, if so
do not access those secure registers.
Signed-off-by: Nsrinidhi kasagar <srinidhi.kasagar@stericsson.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The zero page is read-only, and has its cache state cleared during
boot.  No further maintanence for this page is required.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

page_address() is a function call rather than a macro, and so:

	if (page_address(page))
		do_something(page_address(page));

results in two calls to this function.  This is unnecessary; remove
the duplication.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We had two copies of the wrapper code for VIVT cache flushing - one in
asm/cacheflush.h and one in arch/arm/mm/flush.c.  Reduce this down to
one common copy.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NTomáš Čech <sleep_walker@suse.cz>
Signed-off-by: NEric Miao <eric.y.miao@gmail.com>

Support for the Tauros2 L2 cache controller as used with the PJ1
and PJ4 CPUs.
Signed-off-by: NLennert Buytenhek <buytenh@marvell.com>
Signed-off-by: NSaeed Bishara <saeed@marvell.com>
Signed-off-by: NNicolas Pitre <nico@marvell.com>

The Marvell Dove (88AP510) is a high-performance, highly integrated,
low power SoC with high-end ARM-compatible processor (known as PJ4),
graphics processing unit, high-definition video decoding acceleration
hardware, and a broad range of peripherals.
Signed-off-by: NLennert Buytenhek <buytenh@marvell.com>
Signed-off-by: NSaeed Bishara <saeed@marvell.com>
Signed-off-by: NNicolas Pitre <nico@marvell.com>

On ARMv7, it is invalid to map the same physical address multiple times
with different memory types.  Since system RAM is already mapped as
'memory', subsequent remapping of it must retain this attribute.

However, DMA memory maps it as "strongly ordered".  Fix this by introducing
'pgprot_dmacoherent()' which provides the necessary page table bits for
DMA mappings.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>

It's unnecessary; x86 doesn't do it, and ALSA doesn't require it
anymore.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

This entirely separates the DMA coherent buffer remapping code from
the allocation code, and gets rid of the duplicate copy in the !MMU
section.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

IXP23xx added support for dma_alloc_coherent() for DMA arches with an
exception in dma_alloc_coherent().  This is a subset of what goes on
in __dma_alloc(), and there is no reason why dma_alloc_writecombine()
should not be given the same treatment (except, maybe, that IXP23xx
doesn't use it.)

We can better deal with this by moving the arch_is_coherent() test
inside __dma_alloc() and killing the code duplication.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

No point wrapping the contents of this function with #ifdef CONFIG_MMU
when we can place it and the core_initcall() entirely within the
existing conditional block.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

We effectively have three implementations of dma_free_coherent() mixed up
in the code; the incoherent MMU, coherent MMU and noMMU versions.

The coherent MMU and noMMU versions are actually functionally identical.
The incoherent MMU version is almost the same, but with the additional
step of unmapping the secondary mapping.

Separate out this additional step into __dma_free_remap() and simplify
the resulting dma_free_coherent() code.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

The nommu version of dma_alloc_coherent was using kmalloc/kfree to manage
the memory.  dma_alloc_coherent() is expected to work with a granularity
of a page, so this is wrong.  Fix it by using the helper functions now
provided.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

The coherent architecture dma_alloc_coherent was using kmalloc/kfree to
manage the memory.  dma_alloc_coherent() is expected to work with a
granularity of a page, so this is wrong.  Fix it by using the helper
functions now provided.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

Samsung S5PC1xx SoCs are based on ARM Coretex8, which has 64 bytes of L1
cache line size. Enable proper handling of L1 cache on these SoCs.
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

If running in non-secure mode, enabling this register will fault.
Signed-off-by: NTony Thompson <Anthony.Thompson@arm.com>
Acked-by: NSrinidhi Kasagar <srinidhikasagar@gmail.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Mapping the same memory using two different attributes (memory
type, shareability, cacheability) is unpredictable.  During boot,
we encounter a situation when we're updating the kernel's page
tables which can lead to dirty cache lines existing in the cache
which are subsequently missed.  This causes stack corruption,
and therefore a crash.

Therefore, ensure that the shared and cacheability settings
matches the configuration that will be used later; this together
with the restriction in early_cachepolicy() ensures that we won't
create a mismatch during boot.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Errata 411920 indicates that any "invalidate entire instruction cache"
operation can fail if the right conditions are present.  This is not
limited just to those operations in flush.c, but elsewhere.  Place the
workaround in the already existing __flush_icache_all() function
instead.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This adds a better sched_clock() to the IOP platform,
implemented using its new clocksource support.

Tested on n2100, compile-tested for all plat-iop machines.

[dan.j.williams@intel.com: allow early cp6 access]
Signed-off-by: NMikael Pettersson <mikpe@it.uu.se>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

When SPARSEMEM_EXTREME is enabled, memory_present() wants to use bootmem
to allocate data structures.  However, we call memory_present() after
declaring memory to bootmem, but before we've reserved areas.

This leads to sparsemem data structures being overwritten later in the
kernel's initialization (when slab initializes.)
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We were using GFP_DMA for masks other than 0xffffffff, which is
wrong when some masks are initialized to 0xffffffffffffffff.
This caused such masks to obtain memory from the precious DMA
pool.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Remove the URL listed for Maverick EP9312 since it is not available
and modify the help text appropriately.
Signed-off-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Acked-by: NRyan Mallon <ryan@bluewatersys.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

On ARM, update_mmu_cache() does dcache flush for a page only if
it has a kernel mapping (page_mapping(page) != NULL). The correct
behavior would be to force the flush based on dcache_dirty bit only.

One of the cases where present logic would be a problem is when
a RAM based block device[1] is used as a swap disk. In this case,
we would have in-memory data corruption as shown in steps below:

do_swap_page()
{
    - Allocate a new page (if not already in swap cache)
    - Issue read from swap disk
        - Block driver issues flush_dcache_page()
        - flush_dcache_page() simply sets PG_dcache_dirty bit and does not
          actually issue a flush since this page has no user space mapping yet.
    - Now, if swap disk is almost full, this newly read page is removed
      from swap cache and corrsponding swap slot is freed.
    - Map this page anonymously in user space.
    - update_mmu_cache()
        - Since this page does not have kernel mapping (its not in page/swap
          cache and is mapped anonymously), it does not issue dcache flush
          even if dcache_dirty bit is set by flush_dcache_page() above.

    <user now gets stale data since dcache was never flushed>
}

Same problem exists on mips too.

[1] example:
 - brd (RAM based block device)
 - ramzswap (RAM based compressed swap device)
Signed-off-by: NNitin Gupta <ngupta@vflare.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Seemingly this support was missed when highmem was added, so
DEBUG_HIGHMEM wouldn't have checked the kmap_atomic type.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

If sparsemem is enabled, the start_pfn passed to the free_memmap()
function corresponds to an area of memory not known to the kernel and
pfn_to_page returns a wrong value. The (start_pfn - 1), however, is
known to the kernel.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This is needed because applications using the sys_cacheflush system call
can pass a memory range which isn't mapped yet even though the
corresponding vma is valid. The patch also adds unwinding annotations
for correct backtraces from the coherent_user_range() functions.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

According to the following in arch/arm/mm/fault.c page faults from
kernel mode are invalid if mmap_sem is already held and there is
no exception handler defined for the faulting instruction:

/*
 * As per x86, we may deadlock here.  However, since the kernel only
 * validly references user space from well defined areas of the code,
 * we can bug out early if this is from code which shouldn't.
 */
if (!down_read_trylock(&mm->mmap_sem)) {
	if (!user_mode(regs) && !search_exception_tables(regs->ARM_pc))
		goto no_context;

Since mmap_sem can be held at arbitrary times by another thread this
also means that any page faults from kernel mode are invalid if no
exception handler is defined for them, regardless whether mmap_sem is
held at the time of fault.

To easier detect code that can trigger the above error, add a check
also for the case where mmap_sem is acquired. As this has an overhead
make it a VM debug check.
Signed-off-by: NImre Deak <imre.deak@nokia.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Currently, on ARMv6 and ARMv7, if an application tries to execute
code (or garbage) on non-executable page it hangs. It caused by
incorrect prefetch abort handling. Now every prefetch abort
processes as a translation fault.

To fix this we have to analyze instruction fault status register
to figure out reason why we've got the abort and process it
accordingly.

To make IFSR different from DFSR we set bit 31 which is reserved in
both IFSR and DFSR.

This patch also tries to protect from future hangs on unexpected
exceptions. An application will be killed if unexpected exception
type was received.
Signed-off-by: NKirill A. Shutemov <kirill@shutemov.name>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Instruction fault status register, IFSR, was introduced on ARMv6 to
provide status information about the last insturction fault. It
needed for proper prefetch abort handling.

Now we have three prefetch abort model:

  * legacy - for CPUs before ARMv6. They doesn't provide neither
    IFSR nor IFAR. We simulate IFSR with section translation fault
    status for them to generalize code;
  * ARMv6 - provides IFSR, but not IFAR;
  * ARMv7 - provides both IFSR and IFAR.
Signed-off-by: NKirill A. Shutemov <kirill@shutemov.name>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>