This patch adds support for CMA to dma-mapping subsystem for ARM
architecture. By default a global CMA area is used, but specific devices
are allowed to have their private memory areas if required (they can be
created with dma_declare_contiguous() function during board
initialisation).

Contiguous memory areas reserved for DMA are remapped with 2-level page
tables on boot. Once a buffer is requested, a low memory kernel mapping
is updated to to match requested memory access type.

GFP_ATOMIC allocations are performed from special pool which is created
early during boot. This way remapping page attributes is not needed on
allocation time.

CMA has been enabled unconditionally for ARMv6+ systems.
Signed-off-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
CC: Michal Nazarewicz <mina86@mina86.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Tested-by: NRob Clark <rob.clark@linaro.org>
Tested-by: NOhad Ben-Cohen <ohad@wizery.com>
Tested-by: NBenjamin Gaignard <benjamin.gaignard@linaro.org>
Tested-by: NRobert Nelson <robertcnelson@gmail.com>
Tested-by: NBarry Song <Baohua.Song@csr.com>

WARNING: vmlinux.o(.text+0x111b8): Section mismatch in reference
from the function arm_memory_present() to the function
.init.text:memory_present()
The function arm_memory_present() references
the function __init memory_present().
This is often because arm_memory_present lacks a __init
annotation or the annotation of memory_present is wrong.

WARNING: arch/arm/mm/built-in.o(.text+0x1edc): Section mismatch
in reference from the function alloc_init_pud() to the function
.init.text:alloc_init_section()
The function alloc_init_pud() references
the function __init alloc_init_section().
This is often because alloc_init_pud lacks a __init
annotation or the annotation of alloc_init_section is wrong.
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This line is irritating and wrong when modules are not supported, so
don't show it then.
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

There's no need to include the header twice, so get rid of the
duplicate.
Signed-off-by: NJesper Juhl <jj@chaosbits.net>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

Commit 716a3dc2 (ARM: Add arm_memblock_steal() to allocate memory
away from the kernel) added a function which calls memblock_alloc().
This causes a section conflict:

WARNING: vmlinux.o(.text+0xc614): Section mismatch in reference from the function arm_memblock_steal() to the function .init.text:memblock_alloc()
The function arm_memblock_steal() references
the function __init memblock_alloc().
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Several platforms are now using the memblock_alloc+memblock_free+
memblock_remove trick to obtain memory which won't be mapped in the
kernel's page tables.  Most platforms do this (correctly) in the
->reserve callback.  However, OMAP has started to call these functions
outside of this callback, and this is extremely unsafe - memory will
not be unmapped, and could well be given out after memblock is no
longer responsible for its management.

So, provide arm_memblock_steal() to perform this function, and ensure
that it panic()s if it is used inappropriately.  Convert everyone
over, including OMAP.

As a result, OMAP with OMAP4_ERRATA_I688 enabled will panic on boot
with this change.  Mark this option as BROKEN and make it depend on
BROKEN.  OMAP needs to be fixed, or 137d105d (ARM: OMAP4: Fix
errata i688 with MPU interconnect barriers.) reverted until such
time it can be fixed correctly.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The only function of memblock_analyze() is now allowing resize of
memblock region arrays.  Rename it to memblock_allow_resize() and
update its users.

* The following users remain the same other than renaming.

  arm/mm/init.c::arm_memblock_init()
  microblaze/kernel/prom.c::early_init_devtree()
  powerpc/kernel/prom.c::early_init_devtree()
  openrisc/kernel/prom.c::early_init_devtree()
  sh/mm/init.c::paging_init()
  sparc/mm/init_64.c::paging_init()
  unicore32/mm/init.c::uc32_memblock_init()

* In the following users, analyze was used to update total size which
  is no longer necessary.

  powerpc/kernel/machine_kexec.c::reserve_crashkernel()
  powerpc/kernel/prom.c::early_init_devtree()
  powerpc/mm/init_32.c::MMU_init()
  powerpc/mm/tlb_nohash.c::__early_init_mmu()  
  powerpc/platforms/ps3/mm.c::ps3_mm_add_memory()
  powerpc/platforms/embedded6xx/wii.c::wii_memory_fixups()
  sh/kernel/machine_kexec.c::reserve_crashkernel()

* x86/kernel/e820.c::memblock_x86_fill() was directly setting
  memblock_can_resize before populating memblock and calling analyze
  afterwards.  Call memblock_allow_resize() before start populating.

memblock_can_resize is now static inside memblock.c.
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
Cc: "H. Peter Anvin" <hpa@zytor.com>

memblock_init() initializes arrays for regions and memblock itself;
however, all these can be done with struct initializers and
memblock_init() can be removed.  This patch kills memblock_init() and
initializes memblock with struct initializer.

The only difference is that the first dummy entries don't have .nid
set to MAX_NUMNODES initially.  This doesn't cause any behavior
difference.
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
Cc: "H. Peter Anvin" <hpa@zytor.com>

24aa0788 (memblock, x86: Replace memblock_x86_reserve/free_range()
with generic ones) removed arch/x86/include/asm/memblock.h and dropped
its inclusion from include/linux/memblock.h which breaks other
architectures which depended on the generic memblock.h pulling in the
arch specific one.

However, the proper fix isn't adding back the asm inclusion.  memblock
doesn't have any arch dependent part and doesn't need arch specific
header file and asm/memblock.h files are either practically empty or
contain mostly unrelated arch specific stuff.

* In microblaze, sh, powerpc, sparc and openrisc, asm/memblock.h is
  either empty or just contains unused MEMBLOCK_DBG() macro.  Remove
  them.

* In arm and unicore32, asm/memblock.h contains arch specific stuff.
  Include it directly from its users.  It might be a good idea to
  rename the header file to avoid confusion.
Signed-off-by: NTejun Heo <tj@kernel.org>
Reported-by: N"H. Peter Anvin" <hpa@zytor.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>

Some upcoming changes must know the VMALLOC_START value, which is based
on high_memory, before bootmem_init() is called.

The best location to set it is in sanity_check_meminfo() where the needed
computation is already done, and in the non MMU case it is trivial to do
now that the meminfo array is already sorted at that point.
Signed-off-by: NNicolas Pitre <nicolas.pitre@linaro.org>

The meminfo array has to be sorted before sanity_check_meminfo() in
arch/arm/mm/mmu.c is called for it to work properly.  This also allows
for a simpler find_limits() in arch/arm/mm/init.c.

The sort is moved to arch/arm/kernel/setup.c because that's where the
meminfo array is populated.  Eventually this should be improved upon
to make the memory bank parser a bit more robust against problems
such as overlapping memory ranges.
Signed-off-by: NNicolas Pitre <nicolas.pitre@linaro.org>

These files all make use of one of the EXPORT_SYMBOL variants
or the THIS_MODULE macro.  So they will need <linux/export.h>
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>

The VM subsystem assumes that there are valid memmap entries from
the bank start aligned to MAX_ORDER_NR_PAGES.

On the Ux500 we have a lot of mem=N arguments on the commandline
triggering this bug several times over and causing kernel
oops messages.

Cc: stable@kernel.org
Cc: Michael Bohan <mbohan@codeaurora.org>
Cc: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: NJohan Palsson <johan.palsson@stericsson.com>
Signed-off-by: NRabin Vincent <rabin.vincent@stericsson.com>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

When ARCH_HAS_HOLES_MEMORYMODEL is selected, pfn_valid calls
memblock_is_memory to test validity of a pfn:

> memblock_is_memory(pfn << PAGE_SHIFT);

On LPAE systems this cuts off the top bits, as the shift occurs before
the value is promoted to a phys_addr_t.

This patch replaces the shift with a call to __pfn_to_phys (which casts
pfn to phys_addr_t before shifting), preventing the loss of significant
bits.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This function can be called during boot to increase the size of the consistent
DMA region above it's default value of 2MB. It must be called before the memory
allocator is initialised, i.e. before any core_initcall.
Signed-off-by: NJon Medhurst <tixy@yxit.co.uk>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>

poison_init_mem() used a loop of:

	while ((count = count - 4))

which has 2 problems - an off by one error so that we do one less word
than we should, and the other is that if count == 0 then we loop forever
and poison too much.  On a platform with HAVE_TCM=y but nothing in the
TCM's, this caused corruption and the platform failed to boot.
Acked-by: NStephen Boyd <sboyd@codeaurora.org>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

One less dependency on mach/memory.h.
Signed-off-by: NNicolas Pitre <nicolas.pitre@linaro.org>

Having this value defined at compile time prevents multiple machines with
conflicting definitions to coexist.  Move it to a variable in preparation
for having a per machine value selected at run time.  This is relevant
only when CONFIG_ZONE_DMA is selected.
Signed-off-by: NNicolas Pitre <nicolas.pitre@linaro.org>

ISA_DMA_THRESHOLD has been unused by non-arch code, so lets now get
rid of it from ARM by replacing it with arm_dma_zone_mask.  Move
dma_supported() and dma_set_mask() out of line, and have
dma_supported() check this new variable instead.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Poisoning __init marked memory can be useful when tracking down
obscure memory corruption bugs. Therefore, poison init memory
with 0xe7fddef0 to catch bugs earlier. The poison value is an
undefined instruction in ARM mode and branch to an undefined
instruction in Thumb mode.
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Place the init sections between the text and data sections.  This
means all code is grouped together at the beginning of the kernel
image, and all data is at the end of the image.  This avoids problems
with the 24-bit branch instruction relocations becoming invalid with
large initramfs images.
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Tested-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We can't cope with initrds outside of memory, so check that the
initrd is within some declared memory to the kernel before using
it.  Otherwise we're likely to OOPS during boot.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The "Virtual memory kernel layout" message at startup already prints
.text and .data.  Print .bss too.
Signed-off-by: NRabin Vincent <rabin@rab.in>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

In commit eb33575c ("[ARM] Double check memmap is actually valid with a
memmap has unexpected holes V2"), a new function, memmap_valid_within,
was introduced to mmzone.h so that holes in the memmap which pass
pfn_valid in SPARSEMEM configurations can be detected and avoided.

The fix to this problem checks that the pfn <-> page linkages are
correct by calculating the page for the pfn and then checking that
page_to_pfn on that page returns the original pfn. Unfortunately, in
SPARSEMEM configurations, this results in reading from the page flags to
determine the correct section. Since the memmap here has been freed,
junk is read from memory and the check is no longer robust.

In the best case, reading from /proc/pagetypeinfo will give you the
wrong answer. In the worst case, you get SEGVs, Kernel OOPses and hung
CPUs. Furthermore, ioremap implementations that use pfn_valid to
disallow the remapping of normal memory will break.

This patch allows architectures to provide their own pfn_valid function
instead of using the default implementation used by sparsemem. The
architecture-specific version is aware of the memmap state and will
return false when passed a pfn for a freed page within a valid section.
Acked-by: NMel Gorman <mgorman@suse.de>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Tested-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Architectures that implement their own show_mem() function did not pass
the filter argument to show_free_areas() to appropriately avoid emitting
the state of nodes that are disallowed in the current context.  This patch
now passes the filter argument to show_free_areas() so those nodes are now
avoided.

This patch also removes the show_free_areas() wrapper around
__show_free_areas() and converts existing callers to pass an empty filter.

ia64 emits additional information for each node, so skip_free_areas_zone()
must be made global to filter disallowed nodes and it is converted to use
a nid argument rather than a zone for this use case.
Signed-off-by: NDavid Rientjes <rientjes@google.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: Kyle McMartin <kyle@mcmartin.ca>
Cc: Helge Deller <deller@gmx.de>
Cc: James Bottomley <jejb@parisc-linux.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

If a dtb is passed to the kernel then the kernel needs to iterate
through compiled-in mdescs looking for one that matches and move the
dtb data to a safe location before it gets accidentally overwritten by
the kernel.

This patch creates a new function, setup_machine_fdt() which is
analogous to the setup_machine_atags() created in the previous patch.
It does all the early setup needed to use a device tree machine
description.

v5: - Print warning with neither dtb nor atags are passed to the kernel
    - Fix bug in setting of __machine_arch_type to the selected machine,
      not just the last machine in the list.
Reported-by: NTixy <tixy@yxit.co.uk>
    - Copy command line directly into boot_command_line instead of cmd_line
v4: - Dump some output when a matching machine_desc cannot be found
v3: - Added processing of reserved list.
    - Backed out the v2 change that copied instead of reserved the
      dtb.  dtb is reserved again and the real problem was fixed by
      using alloc_bootmem_align() for early allocation of RAM for
      unflattening the tree.
    - Moved cmd_line and initrd changes to earlier patch to make series
      bisectable.
v2: Changed to save the dtb by copying into an allocated buffer.
    - Since the dtb will very likely be passed in the first 16k of ram
      where the interrupt vectors live, memblock_reserve() is
      insufficient to protect the dtb data.

[based on work originally written by Jeremy Kerr <jeremy.kerr@canonical.com>]
Tested-by: NTony Lindgren <tony@atomide.com>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Acked-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NGrant Likely <grant.likely@secretlab.ca>

The SPARSEMEM code allocates memmap entries only for sections which are
present (i.e. those which contain some valid memory). The membank checks
in free_unused_memmap do not take this into account and can incorrectly
attempt to free memory which is not allocated, resulting in a BUG() in
the bootmem code.

However, if memory is configured as follows:

    |<----section---->|<----hole---->|<----section---->|
    +--------+--------+--------------+--------+--------+
    | bank 0 | unused |              | bank 1 | unused |
    +--------+--------+--------------+--------+--------+

where a bank only occupies part of a section, the memmap allocated for
the remainder of the section *can* be freed.

This patch modifies the checks in free_unused_memmap so that only valid
memmap entries are considered for removal.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Rather than each platform providing its own function to adjust the
zone sizes, use the new ARM_DMA_ZONE_SIZE definition to perform this
adjustment.  This ensures that the actual DMA zone size and the
ISA_DMA_THRESHOLD/MAX_DMA_ADDRESS definitions are consistent with
each other, and moves this complexity out of the platform code.
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add some basic empty infrastructure for DT support on ARM.

v5: - Fix off-by-one error in size calculation of initrd
    - Stop mucking with cmd_line, and load command line from dt into
      boot_command_line instead which matches the behaviour of ATAGS booting
v3: - moved cmd_line export and initrd setup to this patch to make the
      series bisectable.
    - switched to alloc_bootmem_align() for allocation when
      unflattening the device tree.  memblock_alloc() was not the
      right interface.
Signed-off-by: NJeremy Kerr <jeremy.kerr@canonical.com>
Tested-by: NTony Lindgren <tony@atomide.com>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Acked-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NGrant Likely <grant.likely@secretlab.ca>

Commit ddd588b5 ("oom: suppress nodes that are not allowed from
meminfo on oom kill") moved lib/show_mem.o out of lib/lib.a, which
resulted in build warnings on all architectures that implement their own
versions of show_mem():

	lib/lib.a(show_mem.o): In function `show_mem':
	show_mem.c:(.text+0x1f4): multiple definition of `show_mem'
	arch/sparc/mm/built-in.o:(.text+0xd70): first defined here

The fix is to remove __show_mem() and add its argument to show_mem() in
all implementations to prevent this breakage.

Architectures that implement their own show_mem() actually don't do
anything with the argument yet, but they could be made to filter nodes
that aren't allowed in the current context in the future just like the
generic implementation.
Reported-by: NStephen Rothwell <sfr@canb.auug.org.au>
Reported-by: NJames Bottomley <James.Bottomley@hansenpartnership.com>
Suggested-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NDavid Rientjes <rientjes@google.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The unsigned long datatype is not sufficient for mapping physical addresses
>= 4GB.

This patch ensures that the phys_addr_t datatype is used to represent physical
addresses when converting from a PFN.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Disable the initrd if the passed address already overlaps the reserved
region.  This avoids oopses on Netwinders when NeTTrom tells the kernel
that an initrd is located at mem+4MB, but this overlaps the BSS,
resulting in the kernels in-use BSS being freed.

This should be applied to v2.6.37-stable.

Cc: <stable@kernel.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

bootmem_init() no longer makes several uses of the membank
information, so move this into the one remaining called function
which does use it.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Free the high pages using the memblock memory lists - and more
importantly, exclude any memblock allocations in highmem from the
free'd memory.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Switch arm_bootmem_init() to use memblock instead of membank to
free memory into bootmem.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This was missing from the noMMU code, so there was the possibility
of things not working as expected if out of order memory information
was passed.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We need to round memory regions correctly -- specifically, we need to
round reserved region in the more expansive direction (lower limit
down, upper limit up) whereas usable memory regions need to be rounded
in the more restrictive direction (lower limit up, upper limit down).

This introduces two set of inlines:

	memblock_region_memory_base_pfn()
	memblock_region_memory_end_pfn()
	memblock_region_reserved_base_pfn()
	memblock_region_reserved_end_pfn()

Although they are antisymmetric (and therefore are technically
duplicates) the use of the different inlines explicitly documents the
programmer's intention.

The lack of proper rounding caused a bug on ARM, which was then found
to also affect other architectures.
Reported-by: NRussell King <rmk@arm.linux.org.uk>
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
LKML-Reference: <4CB4CDFD.4020105@kernel.org>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Signed-off-by: NH. Peter Anvin <hpa@linux.intel.com>