Jason Gunthorpe reports a build failure when ARM_PATCH_PHYS_VIRT is
not defined:

In file included from arch/arm/include/asm/page.h:163:0,
                 from include/linux/mm_types.h:16,
                 from include/linux/sched.h:24,
                 from arch/arm/kernel/asm-offsets.c:13:
arch/arm/include/asm/memory.h: In function '__virt_to_phys':
arch/arm/include/asm/memory.h:244:40: error: 'PHYS_OFFSET' undeclared (first use in this function)
arch/arm/include/asm/memory.h:244:40: note: each undeclared identifier is reported only once for each function it appears in
arch/arm/include/asm/memory.h: In function '__phys_to_virt':
arch/arm/include/asm/memory.h:249:13: error: 'PHYS_OFFSET' undeclared (first use in this function)

Fixes: ca5a45c0 ("ARM: mm: use phys_addr_t appropriately in p2v and v2p conversions")
Tested-By: NJason Gunthorpe <jgunthorpe@obsidianresearch.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Make sure that inline assembler that expects 'r' operand
receives 32 bit value.

Before this fix in case of CONFIG_ARCH_PHYS_ADDR_T_64BIT and
CONFIG_ARM_PATCH_PHYS_VIRT __phys_to_virt function passed 64 bit
value to __pv_stub inline assembler where 'r' operand is
expected. Compiler behavior in such case is not well specified.
It worked in little endian case, but in big endian case
incorrect code was generated, where compiler confused which
part of 64 bit value it needed to modify. For example BE
snippet looked like this:

N:0x80904E08 : MOV      r2,#0
N:0x80904E0C : SUB      r2,r2,#0x81000000

when LE similar code looked like this

N:0x808FCE2C : MOV      r2,r0
N:0x808FCE30 : SUB      r2,r2,#0xc0, 8 ; #0xc0000000

Note 'r0' register is va that have to be translated into phys

To avoid this situation use explicit cast to 'unsigned long',
which explicitly discard upper part of phys address and convert
value to 32 bit. Also add comment so such cast will not be
removed in the future.
Signed-off-by: NVictor Kamensky <victor.kamensky@linaro.org>
Acked-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Olof Johansson reported:

In file included from arch/arm/include/asm/page.h:163:0,
                 from include/linux/mm_types.h:16,
                 from include/linux/sched.h:24,
                 from arch/arm/kernel/asm-offsets.c:13:
arch/arm/include/asm/memory.h: In function '__virt_to_idmap':
arch/arm/include/asm/memory.h:300:6: error: 'arch_virt_to_idmap' undeclared (first use in this function)

caused by arch_virt_to_idmap being placed inside a different
preprocessor conditional to its user.  Move it along side its user.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The current phys_to_virt patching mechanism works only for 32 bit
physical addresses and this patch extends the idea for 64bit physical
addresses.

The 64bit v2p patching mechanism patches the higher 8 bits of physical
address with a constant using 'mov' instruction and lower 32bits are patched
using 'add'. While this is correct, in those platforms where the lowmem addressable
physical memory spawns across 4GB boundary, a carry bit can be produced as a
result of addition of lower 32bits. This has to be taken in to account and added
in to the upper. The patched __pv_offset and va are added in lower 32bits, where
__pv_offset can be in two's complement form when PA_START < VA_START and that can
result in a false carry bit.

e.g
    1) PA = 0x80000000; VA = 0xC0000000
       __pv_offset = PA - VA = 0xC0000000 (2's complement)

    2) PA = 0x2 80000000; VA = 0xC000000
       __pv_offset = PA - VA = 0x1 C0000000

So adding __pv_offset + VA should never result in a true overflow for (1).
So in order to differentiate between a true carry, a __pv_offset is extended
to 64bit and the upper 32bits will have 0xffffffff if __pv_offset is
2's complement. So 'mvn #0' is inserted instead of 'mov' while patching
for the same reason. Since mov, add, sub instruction are to patched
with different constants inside the same stub, the rotation field
of the opcode is using to differentiate between them.

So the above examples for v2p translation becomes for VA=0xC0000000,
    1) PA[63:32] = 0xffffffff
       PA[31:0] = VA + 0xC0000000 --> results in a carry
       PA[63:32] = PA[63:32] + carry

       PA[63:0] = 0x0 80000000

    2) PA[63:32] = 0x1
       PA[31:0] = VA + 0xC0000000 --> results in a carry
       PA[63:32] = PA[63:32] + carry

       PA[63:0] = 0x2 80000000

The above ideas were suggested by Nicolas Pitre <nico@linaro.org> as
part of the review of first and second versions of the subject patch.

There is no corresponding change on the phys_to_virt() side, because
computations on the upper 32-bits would be discarded anyway.

Cc: Russell King <linux@arm.linux.org.uk>
Reviewed-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NSricharan R <r.sricharan@ti.com>
Signed-off-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>

On some PAE systems (e.g. TI Keystone), memory is above the
32-bit addressable limit, and the interconnect provides an
aliased view of parts of physical memory in the 32-bit addressable
space.  This alias is strictly for boot time usage, and is not
otherwise usable because of coherency limitations. On such systems,
the idmap mechanism needs to take this aliased mapping into account.

This patch introduces virt_to_idmap() and a arch function pointer which
can be populated by platform which needs it. Also populate necessary
idmap spots with now available virt_to_idmap(). Avoided #ifdef approach
to be compatible with multi-platform builds.

Most architecture won't touch it and in that case virt_to_idmap()
fall-back to existing virt_to_phys() macro.

Cc: Russell King <linux@arm.linux.org.uk>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>

Fix remainder types used when converting back and forth between
physical and virtual addresses.

Cc: Russell King <linux@arm.linux.org.uk>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>

VALID_PAGE() has been removed from kernel long time ago,
so fix the comment.
Signed-off-by: NJiang Liu <jiang.liu@huawei.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Nicolas Pitre <nico@linaro.org>
Cc: Stephen Boyd <sboyd@codeaurora.org>
Cc: Giancarlo Asnaghi <giancarlo.asnaghi@st.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

On LPAE machines, PHYS_OFFSET evaluates to a phys_addr_t and this type is
inherited by the PHYS_PFN_OFFSET definition as well.  Consequently, the kernel
build emits warnings of the form:

init/main.c: In function 'start_kernel':
init/main.c:588:7: warning: format '%lx' expects argument of type 'long unsigned int', but argument 2 has type 'phys_addr_t' [-Wformat]

This patch fixes this warning by pinning down the PFN type to unsigned long.
Signed-off-by: NCyril Chemparathy <cyril@ti.com>
Acked-by: NNicolas Pitre <nico@linaro.org>
Tested-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: NSubash Patel <subash.rp@samsung.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch redefines the early boot time use of the R4 register to steal a few
low order bits (ARCH_PGD_SHIFT bits) on LPAE systems.  This allows for up to
38-bit physical addresses.
Signed-off-by: NCyril Chemparathy <cyril@ti.com>
Signed-off-by: NVitaly Andrianov <vitalya@ti.com>
Acked-by: NNicolas Pitre <nico@linaro.org>
Tested-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: NSubash Patel <subash.rp@samsung.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Parts of the virtual memory layout (mainly the modules area) are
described using open-coded immediate values.

Use the SZ_ definitions from linux/sizes.h instead to make the code
clearer.
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We are getting a number of warnings about the use of the deprecated
bus_to_virt function in drivers using the ARM ISA DMA API:

drivers/parport/parport_pc.c: In function 'parport_pc_fifo_write_block_dma':
drivers/parport/parport_pc.c:622:3: warning: 'bus_to_virt' is deprecated
(declared at arch/arm/include/asm/memory.h:253) [-Wdeprecated-declarations]

This is only because that function gets used by the inline
set_dma_addr() helper. We know that any driver for the ISA DMA API
is correctly using the DMA addresses, so we can change this
to use the __bus_to_virt() function instead, which does not warn.

After this, there are no remaining drivers that are used on
any defconfigs on ARM using virt_to_bus or bus_to_virt, with
the exception of the OSS sound driver. That driver is only used
on RiscPC, NetWinder and Shark, so we can set ARCH_NO_VIRT_TO_BUS
on all other platforms and hide the deprecated functions, which
is far more effective than marking them as deprecated, in order
to avoid any new users of that code.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Cc: Russell King <linux@arm.linux.org.uk>

We have received multiple reports of mmap failures when running with a
2:2 vm split. These manifest as either -EINVAL with a non page-aligned
address (ending 0xaaa) or a SEGV, depending on the application. The
issue is commonly observed in children of make, which appears to use
bottom-up mmap (assumedly because it changes the stack rlimit).

Further investigation reveals that this regression was triggered by
394ef640 ("mm: use vm_unmapped_area() on arm architecture"), whereby
TASK_UNMAPPED_BASE is no longer page-aligned for bottom-up mmap, causing
get_unmapped_area to choke on misaligned addressed.

This patch fixes the problem by defining TASK_UNMAPPED_BASE in terms of
TASK_SIZE and explicitly aligns the result to 16M, matching the other
end of the heap.
Acked-by: NNicolas Pitre <nico@linaro.org>
Reported-by: NSteve Capper <steve.capper@arm.com>
Reported-by: NJean-Francois Moine <moinejf@free.fr>
Reported-by: NChristoffer Dall <cdall@cs.columbia.edu>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

With ixp2xxx removed, there are no platforms that define arch_is_coherent,
so the last occurrences of arch_is_coherent can be removed. Any new
platform with coherent i/o should use coherent dma mapping functions.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: NMarek Szyprowski <m.szyprowski@samsung.com>

During the p2v changes, the PHYS_OFFSET #define moved into a
!__ASSEMBLY__ section. This causes a XIP build to fail with

 arch/arm/kernel/head.o: In function 'stext':
 arch/arm/kernel/head.S:146: undefined reference to 'PHYS_OFFSET'

Momentarily leave the #ifndef __ASSEMBLY__ section so we can
define PHYS_OFFSET for all compilation units.
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NAlessandro Rubini <rubini@gnudd.com>
Acked-by: NGiancarlo Asnaghi <giancarlo.asnaghi@st.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Cc: Alan Cox <alan@linux.intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

XIP_VIRT_ADDR is needed for XIP builds and currently only defined for
builds with CONFIG_MMU.

Also provide it for no-MMU builds to make it possible to build an XIP
kernel for MMU-less machines. As these lack an MMU it has to be an
identity mapping.
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Given that we want the default to not have any <mach/memory.h> and given
that there are now fewer cases where it is still provided than the cases
where it is not at this point, this makes sense to invert the logic and
just identify the exception cases.

The word "need" instead of "have" was chosen to construct the config
symbol so not to suggest that having a mach/memory.h file is actually
a feature that one should aim for.
Signed-off-by: NNicolas Pitre <nicolas.pitre@linaro.org>

When the CONFIG_NO_MACH_MEMORY_H symbol is selected by a particular
machine class, the machine specific memory.h include file is no longer
used and can be removed.  In that case the equivalent information can
be obtained dynamically at runtime by enabling CONFIG_ARM_PATCH_PHYS_VIRT
or by specifying the physical memory address at kernel configuration time.

If/when all instances of mach/memory.h are removed then this symbol could
be removed.
Signed-off-by: NNicolas Pitre <nicolas.pitre@linaro.org>

This function can be called during boot to increase the size of the consistent
DMA region above it's default value of 2MB. It must be called before the memory
allocator is initialised, i.e. before any core_initcall.
Signed-off-by: NJon Medhurst <tixy@yxit.co.uk>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>

This code can be removed now that MSM targets no longer need the 16-bit
offsets for P2V.
Signed-off-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ISA_DMA_THRESHOLD has been unused by non-arch code, so lets now get
rid of it from ARM by replacing it with arm_dma_zone_mask.  Move
dma_supported() and dma_set_mask() out of line, and have
dma_supported() check this new variable instead.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Rather than each platform providing its own function to adjust the
zone sizes, use the new ARM_DMA_ZONE_SIZE definition to perform this
adjustment.  This ensures that the actual DMA zone size and the
ISA_DMA_THRESHOLD/MAX_DMA_ADDRESS definitions are consistent with
each other, and moves this complexity out of the platform code.
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The values of ISA_DMA_THRESHOLD and MAX_DMA_ADDRESS are related; one is
the physical/bus address, the other is the virtual address.  Both need
to be kept in step, so rather than having platforms define both, allow
them to define a single macro which sets both of these macros
appropraitely.
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The unsigned long datatype is not sufficient for mapping physical addresses
>= 4GB.

This patch ensures that the address conversion code in asm/memory.h casts
to the correct type when handling physical addresses. The internal v2p
macros only deal with lowmem addresses, so these do not need to be modified.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

MSM's memory is aligned to 2MB, which is more than we can do with our
existing method as we're limited to the upper 8 bits.  Extend this by
using two instructions to 16 bits, automatically selected when MSM is
enabled.
Acked-by: NTony Lindgren <tony@atomide.com>
Reviewed-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Tested-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This idea came from Nicolas, Eric Miao produced an initial version,
which was then rewritten into this.

Patch the physical to virtual translations at runtime.  As we modify
the code, this makes it incompatible with XIP kernels, but allows us
to achieve this with minimal loss of performance.

As many translations are of the form:

	physical = virtual + (PHYS_OFFSET - PAGE_OFFSET)
	virtual = physical - (PHYS_OFFSET - PAGE_OFFSET)

we generate an 'add' instruction for __virt_to_phys(), and a 'sub'
instruction for __phys_to_virt().  We calculate at run time (PHYS_OFFSET
- PAGE_OFFSET) by comparing the address prior to MMU initialization with
where it should be once the MMU has been initialized, and place this
constant into the above add/sub instructions.

Once we have (PHYS_OFFSET - PAGE_OFFSET), we can calculate the real
PHYS_OFFSET as PAGE_OFFSET is a build-time constant, and save this for
the C-mode PHYS_OFFSET variable definition to use.

At present, we are unable to support Realview with Sparsemem enabled
as this uses a complex mapping function, and MSM as this requires a
constant which will not fit in our math instruction.

Add a module version magic string for this feature to prevent
incompatible modules being loaded.
Tested-by: NTony Lindgren <tony@atomide.com>
Reviewed-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Tested-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This uncouple PHYS_OFFSET from the platform definitions, thereby
facilitating run-time computation of the physical memory offset.
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Acked-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Acked-by: NMagnus Damm <damm@opensource.se>
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NJean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
Acked-by: NWan ZongShun <mcuos.com@gmail.com>
Acked-by: NKukjin Kim <kgene.kim@samsung.com>
Acked-by: NEric Miao <eric.y.miao@gmail.com>
Acked-by: NJiandong Zheng <jdzheng@broadcom.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Changing the virt_to_phys() argument to "const volatile void *" avoids
compiler warnings in some situations where this function is used.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NStephen Boyd <sboyd@codeaurora.org>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This changes the TCM handling so that a fixed area is reserved at
0xfffe0000-0xfffeffff for TCM. This areas is used by XScale but
XScale does not have TCM so the mechanisms are mutually exclusive.

This change is needed to make TCM detection more dynamic while
still being able to compile code into it, and is a must for the
unified ARM goals: the current TCM allocation at different places
in memory for each machine would be a nightmare if you want to
compile a single image for more than one machine with TCM so it
has to be nailed down in one place.
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Since we no longer support discontigmem, node is always zero, so
remove this argument.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Everything should now be using sparsemem rather than discontigmem, so
remove the code supporting discontigmem from ARM.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Makes it consistent with VMALLOC_START
Tested-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Signed-off-by: NAndreas Fenkart <andreas.fenkart@streamunlimited.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Adds DMA area to 'virtual memory map' startup message
Tested-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Signed-off-by: NAndreas Fenkart <andreas.fenkart@streamunlimited.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The non-highmem() and the __pfn_to_bus() based page_to_dma() both
compile to the same code, so its pointless having these two different
approaches.  Use the __pfn_to_bus() based version.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-By: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-By: NJamie Iles <jamie@jamieiles.com>
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>

This reverts commit 75f4aa15.

We have a couple of platforms which require non-linear P:V mappings,
so we need these to be overridable.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

On OMAP platforms, some people want to declare to segment up the memory
between the kernel and a separate application such that there is a hole
in the middle of the memory as far as Linux is concerned.  However,
they want to be able to mmap() the hole.

This currently causes problems, because update_mmu_cache() thinks that
there are valid struct pages for the "hole".  Fix this by making
pfn_valid() slightly more expensive, by checking whether the PFN is
contained within the meminfo array.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NKhasim Syed Mohammed <khasim@ti.com>

Modules compiled to Thumb-2 have two additional relocations needing to
be resolved at load time, R_ARM_THM_CALL and R_ARM_THM_JUMP24, for BL
and B.W instructions. The maximum Thumb-2 addressing range is +/-2^24
(+/-16MB) therefore the MODULES_VADDR macro in asm/memory.h is set to
(MODULES_END - 8MB) for the Thumb-2 compiled kernel.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

If a machine class has a custom __virt_to_bus() implementation then it
must provide a __arch_page_to_dma() implementation as well which is
_not_ based on page_address() to support highmem.

This patch fixes existing __arch_page_to_dma() and provide a default
implementation otherwise.  The default implementation for highmem is
based on __pfn_to_bus() which is defined only when no custom
__virt_to_bus() is provided by the machine class.

That leaves only ebsa110 and footbridge which cannot support highmem
until they provide their own __arch_page_to_dma() implementation.
But highmem support on those legacy platforms with limited memory is
certainly not a priority.
Signed-off-by: NNicolas Pitre <nico@marvell.com>

The kmap virtual area borrows a 2MB range at the top of the 16MB area
below PAGE_OFFSET currently reserved for kernel modules and/or the
XIP kernel.  This 2MB corresponds to the range covered by 2 consecutive
second-level page tables, or a single pmd entry as seen by the Linux
page table abstraction.  Because XIP kernels are unlikely to be seen
on systems needing highmem support, there shouldn't be any shortage of
VM space for modules (14 MB for modules is still way more than twice the
typical usage).

Because the virtual mapping of highmem pages can go away at any moment
after kunmap() is called on them, we need to bypass the delayed cache
flushing provided by flush_dcache_page() in that case.

The atomic kmap versions are based on fixmaps, and
__cpuc_flush_dcache_page() is used directly in that case.
Signed-off-by: NNicolas Pitre <nico@marvell.com>