Setting DMA_MAYBE_COHERENT gives a platform the opportunity to select
use of cache ops at boot.
Signed-off-by: NManuel Lauss <manuel.lauss@gmail.com>
Cc: Linux-MIPS <linux-mips@linux-mips.org>
Patchwork: https://patchwork.linux-mips.org/patch/6575/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Allow secondary cores to program their segment control registers
during smp bootstrap code. This enables EVA on Malta SMP
configurations
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Shift System Controller memory mapping to 0x80000000
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Use a Malta specific function to free the init section once the
kernel has booted. When operating in EVA mode, the physical memory
is shifted to 0x80000000. Kernel is loaded into 0x80000000 (virtual)
so the offset between physical and virtual addresses is 0.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

PHYS_OFFSET is used to denote the physical start address of the
first bank of RAM. When the Malta board is in EVA mode, the physical
start address of RAM is shifted to 0x80000000 so it's necessary to use
this macro in order to make the code EVA agnostic.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

The 'ememsize' variable is used to denote the real RAM which is
present on the Malta board. This is different compared to 'memsize'
which is capped to 256MB. The 'ememsize' is used to get the actual
physical memory when setting up the Malta memory layout. This only
makes sense in case the core operates in the EVA mode, and it's
ignored otherwise.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Add a spaces.h file for Malta to override certain memory macros
when operating in EVA mode.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

The Malta board aliases 0x80000000 - 0xffffffff to 0x00000000
- 0x7fffffff ignoring the 256 MB IO hole in 0x10000000.
The physical memory is shifted to 0x80000000 so up to 2GB
can be used. Kuseg is expanded to 3GB (due to board limitations
only 2GB can be accessed) and lowmem (kernel space) is expanded to 2GB.

The Segment Control registers are programmed as follows:

Virtual memory           Physical memory           Mapping
0x00000000 - 0x7fffffff  0x80000000 - 0xfffffffff   MUSUK (kuseg)
0x80000000 - 0x9fffffff  0x00000000 - 0x1ffffffff   MUSUK (kseg0)
0xa0000000 - 0xbf000000  0x00000000 - 0x1ffffffff   MUSUK (kseg1)
0xc0000000 - 0xdfffffff             -                 MK  (kseg2)
0xe0000000 - 0xffffffff             -                 MK  (kseg3)

The location of exception vectors remain the same since 0xbfc00000
(traditional exception base) still maps to 0x1fc00000 physical.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

There is a chance for the secondary cache to have memory
aliases. This can happen if the bootloader is in a non-EVA mode
(or even in EVA mode but with different mapping from the kernel)
and the kernel switching to EVA afterwards. It's best to flush
the icache to avoid having the secondary CPUs fetching stale
data from it.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Use the userspace cache flushing functions if the interrupted
process is a userspace one.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Build EVA specific cache flushing functions (ie cachee).
They will be used by a subsequent patch.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

A core in EVA mode can have any possible segment mapping, so the
default free_initmem_default() function may not always work as expected.
Therefore, add a callback that platforms can use to free up the init section.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

The MIPS *Aptiv family uses bit 28 in Config5 CP0 register to
indicate whether the core supports EVA or not.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

This will allow platforms to use an alternative way to get
the physical address of a symbol.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

When flushing the icache, make sure the address limit is correct
so the appropriate 'cache' instruction will be used. This has no
impact on cores operating in non-eva mode. However, when EVA is
enabled, we ensure that 'cache' will be used instead of 'cachee'.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Add EVA cache flushing functions similar to non-EVA configurations.
Because the cache may or may not contain user virtual addresses, we
need to use the 'cache' or 'cachee' instruction based on whether we
flush the cache on behalf of kernel or user respectively.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Build code to invalidate an address range in the  instruction cache
using the Hit Invalidate cache operation.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

EVA does not have FPU specific instructions for reading or writing
FPU registers from userspace memory.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

A MIPS specific csum_and_copy_from_user function is necessary because
the generic one from include/net/checksum.h will not work for EVA.
This is because the generic one will link to symbols from lib/checksum.c
which are not EVA aware.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

In EVA mode, different instructions need to be used to read/write
from kernel and userland. In non-EVA mode, there is no functional
difference. The current address limit is checked to decide the
type of operation that will be performed.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Use EVA specific functions to read and write data to
user address space.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

In preparation for EVA support, we use a macro to build the
__csum_partial_copy_user main code so it can be shared across
multiple implementations. EVA uses the same code but it replaces
the load/store/prefetch instructions with the EVA specific ones
therefore using a macro avoids unnecessary code duplications.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Each load/store macro always adds an entry to the __ex_table
using the EXC macro. There are cases where a load instruction may
never fail such as when we are sure the load happens in the kernel
address space. Therefore, we merge these the EXC and LOADX/STOREX
macros into a single one. We also expand the argument list in the EXC
macro to make the macro more flexible. The extra 'type' argument is not
used by this commit, but it will be used when EVA support is added to
memcpy.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

The 'copy_user' symbol can be used to copy from or to
userland so we will use two different symbols for these
operations. This makes no difference in the existing code,
but when the core is operating in EVA mode, different instructions
need to be used to read and write to userland address space.
The old function has also been renamed to 'copy_kernel' to denote
that it is suitable for copy data to and from kernel space.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Handle unaligned accesses when we access userspace memory
EVA mode.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Use the load/store instruction wrappers from asm/asm.h to
perform such operations when operating in EVA mode.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

The str*_user functions are used to securely access NULL terminated
strings from userland. Therefore, it's necessary to use the appropriate
EVA function. However, if the string is in kernel space, then the normal
instructions are being used to access it. The __str*_kernel_asm and
__str*_user_asm symbols are the same for non-EVA mode so there is no
functional change for the non-EVA kernels.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Use the EVA specific functions from memcpy.S to perform
userspace operations. When get_fs() == get_ds() the usual load/store
instructions are used because the destination address is located in
the kernel address space region. Otherwise, the EVA specifc load/store
instructions are used which will go through th TLB to perform the virtual
to physical translation for the userspace address.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

The {get,put}_user_asm functions can be used to load data from
kernel or the user address space so rename them to avoid
confusion.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Use the EVA instruction wrappers from asm.h to perform
read/write operations from userland.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

ulb, ulh, ulw are macros which emulate unaligned access for MIPS.
However, no such macros exist for EVA mode, so the only way to do
EVA unaligned accesses is in the ADE exception handler. As a result
of which, disable these macros for EVA.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Similar to __get_user_* functions, move common code to
__put_user_*_common so it can be shared among similar users.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

In preparation for EVA support, an instruction argument is needed
for the __get_user_asm{,_ll32} functions to allow instruction overrides in
EVA mode. Even though EVA only works for MIPS 32-bit, both codepaths are
changed (32-bit and 64-bit) for consistency reasons.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Build the __bzero function using the EVA load/store instructions
when operating in the EVA mode. This function is only used when
accessing user code so there is no need to build two distinct symbols
for user and kernel operations respectively.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Build the __bzero symbol using a macor. In EVA mode we will
need to use similar code to do the userspace load operations so
it is better if we use a macro to avoid code duplications.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>