The nommu version of dma_alloc_coherent was using kmalloc/kfree to manage
the memory.  dma_alloc_coherent() is expected to work with a granularity
of a page, so this is wrong.  Fix it by using the helper functions now
provided.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

The coherent architecture dma_alloc_coherent was using kmalloc/kfree to
manage the memory.  dma_alloc_coherent() is expected to work with a
granularity of a page, so this is wrong.  Fix it by using the helper
functions now provided.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NGreg Ungerer <gerg@uclinux.org>

If running in non-secure mode, enabling this register will fault.
Signed-off-by: NTony Thompson <Anthony.Thompson@arm.com>
Acked-by: NSrinidhi Kasagar <srinidhikasagar@gmail.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Mapping the same memory using two different attributes (memory
type, shareability, cacheability) is unpredictable.  During boot,
we encounter a situation when we're updating the kernel's page
tables which can lead to dirty cache lines existing in the cache
which are subsequently missed.  This causes stack corruption,
and therefore a crash.

Therefore, ensure that the shared and cacheability settings
matches the configuration that will be used later; this together
with the restriction in early_cachepolicy() ensures that we won't
create a mismatch during boot.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Errata 411920 indicates that any "invalidate entire instruction cache"
operation can fail if the right conditions are present.  This is not
limited just to those operations in flush.c, but elsewhere.  Place the
workaround in the already existing __flush_icache_all() function
instead.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

When SPARSEMEM_EXTREME is enabled, memory_present() wants to use bootmem
to allocate data structures.  However, we call memory_present() after
declaring memory to bootmem, but before we've reserved areas.

This leads to sparsemem data structures being overwritten later in the
kernel's initialization (when slab initializes.)
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We were using GFP_DMA for masks other than 0xffffffff, which is
wrong when some masks are initialized to 0xffffffffffffffff.
This caused such masks to obtain memory from the precious DMA
pool.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Remove the URL listed for Maverick EP9312 since it is not available
and modify the help text appropriately.
Signed-off-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Acked-by: NRyan Mallon <ryan@bluewatersys.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

On ARM, update_mmu_cache() does dcache flush for a page only if
it has a kernel mapping (page_mapping(page) != NULL). The correct
behavior would be to force the flush based on dcache_dirty bit only.

One of the cases where present logic would be a problem is when
a RAM based block device[1] is used as a swap disk. In this case,
we would have in-memory data corruption as shown in steps below:

do_swap_page()
{
    - Allocate a new page (if not already in swap cache)
    - Issue read from swap disk
        - Block driver issues flush_dcache_page()
        - flush_dcache_page() simply sets PG_dcache_dirty bit and does not
          actually issue a flush since this page has no user space mapping yet.
    - Now, if swap disk is almost full, this newly read page is removed
      from swap cache and corrsponding swap slot is freed.
    - Map this page anonymously in user space.
    - update_mmu_cache()
        - Since this page does not have kernel mapping (its not in page/swap
          cache and is mapped anonymously), it does not issue dcache flush
          even if dcache_dirty bit is set by flush_dcache_page() above.

    <user now gets stale data since dcache was never flushed>
}

Same problem exists on mips too.

[1] example:
 - brd (RAM based block device)
 - ramzswap (RAM based compressed swap device)
Signed-off-by: NNitin Gupta <ngupta@vflare.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Seemingly this support was missed when highmem was added, so
DEBUG_HIGHMEM wouldn't have checked the kmap_atomic type.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

If sparsemem is enabled, the start_pfn passed to the free_memmap()
function corresponds to an area of memory not known to the kernel and
pfn_to_page returns a wrong value. The (start_pfn - 1), however, is
known to the kernel.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This is needed because applications using the sys_cacheflush system call
can pass a memory range which isn't mapped yet even though the
corresponding vma is valid. The patch also adds unwinding annotations
for correct backtraces from the coherent_user_range() functions.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

According to the following in arch/arm/mm/fault.c page faults from
kernel mode are invalid if mmap_sem is already held and there is
no exception handler defined for the faulting instruction:

/*
 * As per x86, we may deadlock here.  However, since the kernel only
 * validly references user space from well defined areas of the code,
 * we can bug out early if this is from code which shouldn't.
 */
if (!down_read_trylock(&mm->mmap_sem)) {
	if (!user_mode(regs) && !search_exception_tables(regs->ARM_pc))
		goto no_context;

Since mmap_sem can be held at arbitrary times by another thread this
also means that any page faults from kernel mode are invalid if no
exception handler is defined for them, regardless whether mmap_sem is
held at the time of fault.

To easier detect code that can trigger the above error, add a check
also for the case where mmap_sem is acquired. As this has an overhead
make it a VM debug check.
Signed-off-by: NImre Deak <imre.deak@nokia.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Currently, on ARMv6 and ARMv7, if an application tries to execute
code (or garbage) on non-executable page it hangs. It caused by
incorrect prefetch abort handling. Now every prefetch abort
processes as a translation fault.

To fix this we have to analyze instruction fault status register
to figure out reason why we've got the abort and process it
accordingly.

To make IFSR different from DFSR we set bit 31 which is reserved in
both IFSR and DFSR.

This patch also tries to protect from future hangs on unexpected
exceptions. An application will be killed if unexpected exception
type was received.
Signed-off-by: NKirill A. Shutemov <kirill@shutemov.name>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Instruction fault status register, IFSR, was introduced on ARMv6 to
provide status information about the last insturction fault. It
needed for proper prefetch abort handling.

Now we have three prefetch abort model:

  * legacy - for CPUs before ARMv6. They doesn't provide neither
    IFSR nor IFAR. We simulate IFSR with section translation fault
    status for them to generalize code;
  * ARMv6 - provides IFSR, but not IFAR;
  * ARMv7 - provides both IFSR and IFAR.
Signed-off-by: NKirill A. Shutemov <kirill@shutemov.name>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Commit 1522ac3e
("Fix virtual to physical translation macro corner cases")
breaks the end of memory check in valid_phys_addr_range().
The modified expression results in the apparent /dev/mem size
being 2 bytes smaller than what it actually is.

This patch reworks the expression to correctly check the address,
while maintaining use of a valid address to __pa().
Signed-off-by: NGreg Ungerer <gerg@uclinux.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We suffer an unfortunate combination of "features" which makes highmem
support on platforms without hardware TLB maintainence broadcast difficult:

- we need kmap_high_get() support for DMA cache coherence
- this requires kmap_high() to take a spinlock with IRQs disabled
- kmap_high() occasionally calls flush_all_zero_pkmaps() to clear
  out old mappings
- flush_all_zero_pkmaps() calls flush_tlb_kernel_range(), which
  on s/w IPI'd systems eventually calls smp_call_function_many()
- smp_call_function_many() must not be called with IRQs disabled:

WARNING: at kernel/smp.c:380 smp_call_function_many+0xc4/0x240()
Modules linked in:
Backtrace:
[<c00306f0>] (dump_backtrace+0x0/0x108) from [<c0286e6c>] (dump_stack+0x18/0x1c)
 r6:c007cd18 r5:c02ff228 r4:0000017c
[<c0286e54>] (dump_stack+0x0/0x1c) from [<c0053e08>] (warn_slowpath_common+0x50/0x80)
[<c0053db8>] (warn_slowpath_common+0x0/0x80) from [<c0053e50>] (warn_slowpath_null+0x18/0x1c)
 r7:00000003 r6:00000001 r5:c1ff4000 r4:c035fa34
[<c0053e38>] (warn_slowpath_null+0x0/0x1c) from [<c007cd18>] (smp_call_function_many+0xc4/0x240)
[<c007cc54>] (smp_call_function_many+0x0/0x240) from [<c007cec0>] (smp_call_function+0x2c/0x38)
[<c007ce94>] (smp_call_function+0x0/0x38) from [<c005980c>] (on_each_cpu+0x1c/0x38)
[<c00597f0>] (on_each_cpu+0x0/0x38) from [<c0031788>] (flush_tlb_kernel_range+0x50/0x58)
 r6:00000001 r5:00000800 r4:c05f3590
[<c0031738>] (flush_tlb_kernel_range+0x0/0x58) from [<c009c600>] (flush_all_zero_pkmaps+0xc0/0xe8)
[<c009c540>] (flush_all_zero_pkmaps+0x0/0xe8) from [<c009c6b4>] (kmap_high+0x8c/0x1e0)
[<c009c628>] (kmap_high+0x0/0x1e0) from [<c00364a8>] (kmap+0x44/0x5c)
[<c0036464>] (kmap+0x0/0x5c) from [<c0109dfc>] (cramfs_readpage+0x3c/0x194)
[<c0109dc0>] (cramfs_readpage+0x0/0x194) from [<c0090c14>] (__do_page_cache_readahead+0x1f0/0x290)
[<c0090a24>] (__do_page_cache_readahead+0x0/0x290) from [<c0090ce4>] (ra_submit+0x30/0x38)
[<c0090cb4>] (ra_submit+0x0/0x38) from [<c0089384>] (filemap_fault+0x3dc/0x438)
 r4:c1819988
[<c0088fa8>] (filemap_fault+0x0/0x438) from [<c009d21c>] (__do_fault+0x58/0x43c)
[<c009d1c4>] (__do_fault+0x0/0x43c) from [<c009e8cc>] (handle_mm_fault+0x104/0x318)
[<c009e7c8>] (handle_mm_fault+0x0/0x318) from [<c0033c98>] (do_page_fault+0x188/0x1e4)
[<c0033b10>] (do_page_fault+0x0/0x1e4) from [<c0033ddc>] (do_translation_fault+0x7c/0x84)
[<c0033d60>] (do_translation_fault+0x0/0x84) from [<c002b474>] (do_DataAbort+0x40/0xa4)
 r8:c1ff5e20 r7:c0340120 r6:00000805 r5:c1ff5e54 r4:c03400d0
[<c002b434>] (do_DataAbort+0x0/0xa4) from [<c002bcac>] (__dabt_svc+0x4c/0x60)
...

So we disable highmem support on these systems.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Makes code futureproof against the impending change to mm->cpu_vm_mask.

It's also a chance to use the new cpumask_ ops which take a pointer
(the older ones are deprecated, but there's no hurry for arch code).
Signed-off-by: NRusty Russell <rusty@rustcorp.com.au>

Commit 96177299 ("Drop free_pages()")
modified nr_free_pages() to return 'unsigned long' instead of 'unsigned
int'.  This made the casts to 'unsigned long' in most callers superfluous,
so remove them.

[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: NGeert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Reviewed-by: NChristoph Lameter <cl@linux-foundation.org>
Acked-by: NIngo Molnar <mingo@elte.hu>
Acked-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NDavid S. Miller <davem@davemloft.net>
Acked-by: NKyle McMartin <kyle@mcmartin.ca>
Acked-by: NWANG Cong <xiyou.wangcong@gmail.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Haavard Skinnemoen <hskinnemoen@atmel.com>
Cc: Mikael Starvik <starvik@axis.com>
Cc: "Luck, Tony" <tony.luck@intel.com>
Cc: Hirokazu Takata <takata@linux-m32r.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: David Howells <dhowells@redhat.com>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Chris Zankel <zankel@tensilica.com>
Cc: Michal Simek <monstr@monstr.eu>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

ARMv6 introduces non-executable mappings, which can cause prefetch aborts
when an attempt is made to execute from such a mapping.  Currently, this
causes us to loop in the page fault handler since we don't correctly
check for proper permissions.

Fix this by checking that VMAs have VM_EXEC set for prefetch aborts.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Since we get notified separately about prefetch aborts, which may be
permission faults, we need to check for appropriate access permissions
when handling a fault.  This patch prepares us for doing this by
separating out the access error checking.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This adds the TCM interface to Linux, when active, it will
detect and report TCM memories and sizes early in boot if
present, introduce generic TCM memory handling, provide a
generic TCM memory pool and select TCM memory for the U300
platform.

See the Documentation/arm/tcm.txt for documentation.
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Currently kernel believes that all ARM CPUs have L1_CACHE_SHIFT == 5.
It's not true at least for CPUs based on Cortex-A8.

List of CPUs with cache line size != 32 should be expanded later.
Signed-off-by: NKirill A. Shutemov <kirill@shutemov.name>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Due to problems at cam.org, my nico@cam.org email address is no longer
valid.  FRom now on, nico@fluxnic.net should be used instead.
Signed-off-by: NNicolas Pitre <nico@fluxnic.net>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

On OMAP platforms, some people want to declare to segment up the memory
between the kernel and a separate application such that there is a hole
in the middle of the memory as far as Linux is concerned.  However,
they want to be able to mmap() the hole.

This currently causes problems, because update_mmu_cache() thinks that
there are valid struct pages for the "hole".  Fix this by making
pfn_valid() slightly more expensive, by checking whether the PFN is
contained within the meminfo array.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NKhasim Syed Mohammed <khasim@ti.com>

Let's suppose a highmem page is kmap'd with kmap().  A pkmap entry is
used, the page mapped to it, and the virtual cache is dirtied.  Then
kunmap() is used which does virtually nothing except for decrementing a
usage count.

Then, let's suppose the _same_ page gets mapped using kmap_atomic().
It is therefore mapped onto a fixmap entry instead, which has a
different virtual address unaware of the dirty cache data for that page
sitting in the pkmap mapping.

Fortunately it is easy to know if a pkmap mapping still exists for that
page and use it directly with kmap_atomic(), thanks to kmap_high_get().

And actual testing with a printk in the added code path shows that this
condition is actually met *extremely* frequently.  Seems that we've been
quite lucky that things have worked so well with highmem so far.

Cc: stable@kernel.org
Signed-off-by: NNicolas Pitre <nico@marvell.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

In xdr_partial_copy_from_skb() there is that sequence:

		kaddr = kmap_atomic(*ppage, KM_SKB_SUNRPC_DATA);
		[...]
		flush_dcache_page(*ppage);
		kunmap_atomic(kaddr, KM_SKB_SUNRPC_DATA);

Mixing flush_dcache_page() and kmap_atomic() is a bit odd,
especially since kunmap_atomic() must deal with cache issues
already.  OTOH the non-highmem case must use flush_dcache_page()
as kunmap_atomic() becomes a no op with no cache maintenance.

Problem is that with highmem the implementation of kmap_atomic()
doesn't set page->virtual, and page_address(page) returns 0 in
that case. Here flush_dcache_page() calls __flush_dcache_page()
which calls __cpuc_flush_dcache_page(page_address(page)) resulting
in a kernel oops.

None of the kmap_atomic() implementations uses set_page_address().
Hence we can assume page_address() is always expected to return 0 in
that case. Let's conditionally call __cpuc_flush_dcache_page() only
when the page address is non zero, and perform that test only when
highmem is configured.
Signed-off-by: NNicolas Pitre <nico@marvell.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add the ARM implementation of highpte, which allows PTE tables to be
placed in highmem.  Unfortunately, we do not offer highpte support
when support for L2 cache is enabled.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Currently, highmem is selectable, and you can request an increased
vmalloc area.  However, none of this has any effect on the memory
layout since a patch in the highmem series was accidentally dropped.
Moreover, even if you did want highmem, all memory would still be
registered as lowmem, possibly resulting in overflow of the available
virtual mapping space.

The highmem boundary is determined by the highest allowed beginning
of the vmalloc area, which depends on its configurable minimum size
(see commit 60296c71 for details on
this).

We should create mappings and initialize bootmem only for low memory,
while the zone allocator must still be told about highmem.

Currently, memory nodes which are completely located in high memory
are not supported.  This is not a huge limitation since systems
relying on highmem support are unlikely to have discontiguous memory
with large holes.

[ A similar patch was meant to be merged before commit 5f0fbf9e
  and be available  in Linux v2.6.30, however some git rebase screw-up
  of mine dropped the first commit of the series, and that goofage
  escaped testing somehow as well. -- Nico ]
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Reviewed-by: NNicolas Pitre <nico@marvell.com>

When building with !MMU, task_struct is not defined. Just include the
relevant file.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

ARMv7-R profile CPUs do not have these registers.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

This is needed for the struct meminfo definition.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>