Each load/store macro always adds an entry to the __ex_table
using the EXC macro. There are cases where a load instruction may
never fail such as when we are sure the load happens in the kernel
address space. Therefore, we merge these the EXC and LOADX/STOREX
macros into a single one. We also expand the argument list in the EXC
macro to make the macro more flexible. The extra 'type' argument is not
used by this commit, but it will be used when EVA support is added to
memcpy.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

The 'copy_user' symbol can be used to copy from or to
userland so we will use two different symbols for these
operations. This makes no difference in the existing code,
but when the core is operating in EVA mode, different instructions
need to be used to read and write to userland address space.
The old function has also been renamed to 'copy_kernel' to denote
that it is suitable for copy data to and from kernel space.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

The csum_partial implementation contain optimalizations for the MIPS R2
instruction set. This optimization is never enabled however because the
if directive uses the CPU_MIPSR2 constant which is not defined anywhere.

Use the CONFIG_CPU_MIPSR2 constant instead.
Signed-off-by: NGabor Juhos <juhosg@openwrt.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/4971/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Having received another series of whitespace patches I decided to do this
once and for all rather than dealing with this kind of patches trickling
in forever.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

They possibly silence meaningful warnings ...
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Use unsigned loads to avoid possible misscalculation of IP checksums.  This
bug was instruced in f761106cd728bcf65b7fe161b10221ee00cf7132 (lmo) /
ed99e2bc (kernel.org).

[Original fix by Atsushi.  Improved instruction scheduling and fix for
unaligned unsigned load by me -- Ralf]
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

These symbols appear in oprofile output, stacktraces and similar but only
make the output harder to read.  Many identical symbol names such as
"both_aligned" were also being used in multiple source files making it
impossible to see which file actually was meant.  So let's get rid of them.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

 This complements the generic R4000/R4400 errata workaround code and adds 
bits for the daddiu problem.  In most places it just modifies handwritten 
assembly code so that the assembler is allowed to use a temporary register 
as daddiu may now be treated as a macro that expands to a sequence of li 
and daddu.  It is the AT register or, where AT is unavailable or used 
explicitly for another purpose, an explicitly-named register is selected, 
using the .set at=<reg> feature added recently to gas.  This feature is 
only used if CONFIG_CPU_DADDI_WORKAROUNDS has been set, so if the 
workaround remains disabled, the required version of binutils stays 
unchanged.

 Similarly, daddiu instructions put in branch delay slots in noreorder 
fragments are now taken out of them and the assembler is allowed to 
reorder them itself as possible (which it does making the whole idea of 
scheduling them into delay slots manually questionable).

 Also in the very few places where such a simple conversion was not 
possible, a handcoded longer sequence is implemented.

 Other than that there are changes to code responsible for building the 
TLB fault and page clear/copy handlers to avoid daddiu as appropriate.  
These are only effective if the erratum is verified to be present at the 
run time.

 Finally there is a trivial update to __delay(), because it uses daddiu in 
a branch delay slot.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Implement optimized asm version of csum_partial_copy_nocheck,
csum_partial_copy_from_user and csum_and_copy_to_user which can do
calculate and copy in parallel, based on memcpy.S.
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Make csum_partial 64-bit powered.
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Delete dead codes at end of the function and move small_csumcopy
there.  This makes some labels (maybe_end_cruft, small_memcpy,
end_bytes, out) needless and eliminates some branches.
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Use standard o32 register name instead of T0, T1, etc, like memcpy.S.
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The 32-bit version and 64-bit version are almost equal.  Unify them.  This
makes further improvements (for example, copying with parallel, supporting
PREFETCH, etc.) easier.
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>