Commit c8a34581 ("MIPS: Emulate the BC1{EQ,NE}Z FPU instructions")
added support for emulating the new R6 BC1{EQ,NE}Z branches but it missed
the case where the instruction that caused the exception was not on a DS.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>
Fixes: c8a34581 ("MIPS: Emulate the BC1{EQ,NE}Z FPU instructions")
Cc: <stable@vger.kernel.org> # 4.0+
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/10738/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The mfhc/mthc instructions are supported on MIPS R6 so emulate
them if needed.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.0+
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/10737/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix indentation for the final 'else' blocks.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/10735/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The double format (d_fmt) case uses an opening bracket which then
closes at the end of the word format (w_fmt). This can be rather confusing
so add the closing bracket at the end of the d_fmt case and use another one
for the w_fmt one.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/10733/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Commit 5f9f41c4 ("MIPS: kernel: Prepare
the JR instruction for emulation on MIPS R6") added support for
emulating the JR instruction on MIPS R6 cores but that introduced a bug
which could be triggered when hitting a JALR opcode because the code used
the wrong field in the 'r_format' struct to determine the instruction
opcode. This lead to crashes because an emulated JALR instruction was
treated as a JR one when the R6 emulator was turned off.

Fixes: 5f9f41c4 ("MIPS: kernel: Prepare the JR instruction for emulation on MIPS R6")
Cc: <stable@vger.kernel.org> # 4.0+
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: stable@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/10583/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Commits f1b44067 ("MIPS: Emulate the
new MIPS R6 B{L,G}T{Z,}{AL,}C instructions") and commit
a8ff66f5 ("MIPS: Emulate the new MIPS
R6 B{L,G}E{Z,}{AL,}C instructions") added support for emulating various
branch compact instructions. However, it missed the case for those which
use the old BLEZL and BGTZL opcodes leading to random crashes when the R6
emulator is disabled. We fix this by ensuring that the 'rt' field is not
zero which is always true for these branch compact instructions.

Fixes: f1b44067 ("MIPS: Emulate the new MIPS R6 B{L,G}T{Z,}{AL,}C instructions")
Fixes: a8ff66f5 ("MIPS: Emulate the new MIPS R6 B{L,G}E{Z,}{AL,}C instructions")
Cc: <stable@vger.kernel.org> # 4.0+
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: Markos Chandras <markos.chandras@imgtec.com>
Patchwork: https://patchwork.linux-mips.org/patch/10582/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix "BUG: using smp_processor_id() in preemptible" reported in accesses
to thread's FPU defaults: the value to initialise FSCR to at program
startup, the FCSR r/w mask and the contents of FIR in full FPU
emulation, removing a regression introduced with 9b26616c [MIPS: Respect
the ISA level in FCSR handling] and f6843626 [MIPS: math-emu: Set FIR
feature flags for full emulation].

Use `boot_cpu_data' to obtain the data from, following the approach that
`cpu_has_*' macros take and avoiding the call to `smp_processor_id' made
in the reference to `current_cpu_data'.  The contents of FSCR have to be
consistent across processors in an SMP system, the settings there must
not change as a thread is migrated across processors.  And the contents
of FIR are guaranteed to be consistent in FPU emulation, by definition.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Tested-by: NEzequiel Garcia <ezequiel.garcia@imgtec.com>
Tested-by: NPaul Martin <paul.martin@codethink.co.uk>
Cc: Markos Chandras <Markos.Chandras@imgtec.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/10030/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Define the central place the default FCSR value is set from, initialised
in `cpu_probe'.  Determine the FCSR mask applied to values written to
the register with CTC1 in the full emulation mode and via ptrace(2),
according to the ISA level of processor hardware or the writability of
bits 31:18 if actual FPU hardware is used.

Software may rely on FCSR bits whose functions our emulator does not
implement, so it should not allow them to be set or software may get
confused.  For ptrace(2) it's just sanity.

[ralf@linux-mips.org: Fixed double inclusion of <asm/current.h>.]
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9711/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The ABS.fmt and NEG.fmt instructions have been specified as arithmetic
in the MIPS architecture, which in particular implies handling NaN data
in the usual way with qNaN bit patterns propagated unchanged and sNaN
bit patterns signalling the usual IEEE 754 Invalid Operation exception
and quieted by default.

A series of changes applied over time to our implementation:

c5033d78 [MIPS] ieee754[sd]p_neg workaround
cea2be44 MIPS: Fix abs.[sd] and neg.[sd] emulation for NaN operands

has led to the current situation where the sign bit is updated according
to the operation requested even for NaN inputs.  This is according to
these commits a workaround so that broken binaries produced by GCC
disregarding the properties of these instructions have a chance to work.

For sNaN inputs this remains within IEEE Std 754 as the standard leaves
the choice of output qNaN bit patterns produced under the default
Invalid Operation exception handling for individual sNaN input bit
patterns to implementer's discretion, even though it still recommends as
much NaN input information to be preserved in NaN outputs.

For qNaN inputs however it violates the standard as it requires a qNaN
input bit patterns to propagate unchanged to output.

This is also unlike real MIPS FPU hardware behaves where sNaN and/or
qNaN processing has been fully implemented with no Unimplemented
Operation exception signalled.  Such hardware propagates any input qNaN
bit pattern unchanged.  It also quiets any input sNaN bit pattern in an
implementer-specific manner, for example the MIPS 74Kf processor returns
the default qNaN pattern with the sign bit always clear and the Broadcom
SB-1 and BMIPS5000 processors propagate the input sNaN bit pattern with
the sign bit unchanged and the quiet bit first cleared in the trailing
significand field and then the next lower bit set if clearing the quiet
bit left the field with no other bit set.

Especially the latter observation indicates the limited usefulness of
the workaround as it will cover many hardware configurations, but not
all of them, only making it harder to discover such broken binaries that
need to be recompiled with GCC told to avoid the use of ABS.fmt and
NEG.fmt instructions where non-arithmetic semantics is required by the
algorithm used.

Revert the damage done by the series of changes then, and take the
opportunity to simplify implementation by calling `ieee754dp_sub' and
`ieee754dp_add' as required and also the rounding mode set towards -Inf
temporarily so that the sign of 0 is correctly handled.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9710/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Define IEEE 754-2008 feature control bits: FIR.HAS2008, FCSR.ABS2008 and
FCSR.NAN2008, and update the `_ieee754_csr' structure accordingly.

For completeness define FIR.UFRP too.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9709/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Implement the FCCR, FEXR and FENR "shadow" FPU registers for the
architecture levels that include them, for the CFC1 and CTC1
instructions in the full emulation mode.

For completeness add macros for the CP1 UFR and UNFR registers too, no
actual implementation though.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9708/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Implement FIR feature flags in the FPU emulator according to features
supported and architecture level requirements.  The W, L and F64 bits
have only been added at level #2 even though the features they refer to
were also included with the MIPS64r1 ISA and the W fixed-point format
also with the MIPS32r1 ISA.

This is only relevant for the full emulation mode and the emulated CFC1
instruction as well as ptrace(2) accesses.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9707/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Complement 593d33fe [MIPS: math-emu: Move various objects into an ar
library.] and also move sp_tlong.o, sp_flong.o, dp_tlong.o, and
dp_flong.o into an `ar' library.  These objects implement long
fixed-point format support that can be omitted from MIPS I, MIPS II and
MIPS32r1 configurations.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9702/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Restore EPC at the branch whose delay slot is emulated if the delay-slot
instruction signals.  This is so that code in `fpu_emulator_cop1Handler'
does not see EPC having advanced and mistakenly successfully resume
userland execution from the location at the branch target in that case.
Restoring EPC guarantees an immediate exit from the emulation loop and
if EPC hasn't advanced at all since entering the loop, also issuing the
signal reported by the delay-slot instruction.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9701/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Correct ISA requirements for floating-point instructions:

* the CU3 exception signifies a real COP3 instruction in MIPS I & II,

* the BC1FL and BC1TL instructions are not supported in MIPS I,

* the SQRT.fmt instructions are indeed supported in MIPS II,

* the LDC1 and SDC1 instructions are indeed supported in MIPS32r1,

* the CEIL.W.fmt, FLOOR.W.fmt, ROUND.W.fmt and TRUNC.W.fmt instructions
  are indeed supported in MIPS32,

* the CVT.L.fmt and CVT.fmt.L instructions are indeed supported in
  MIPS32r2 and MIPS32r6,

* the CEIL.L.fmt, FLOOR.L.fmt, ROUND.L.fmt and TRUNC.L.fmt instructions
  are indeed supported in MIPS32r2 and MIPS32r6,

* the RSQRT.fmt and RECIP.fmt instructions are indeed supported in
  MIPS64r1,

Also simplify conditionals for MIPS III and MIPS IV FPU instructions and
the handling of the MOVCI minor opcode.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9700/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Correct a cache coherency regression introduced with be1664c4 [Another
round of fixes for the fp emulator.] for the emulation frame used in
delay-slot emulation.

Two instructions are copied into the frame and as from the commit
referred a cache synchronisation call is made for the second instruction
aka `badinst' of the two only.  The `flush_cache_sigtramp' interface is
reused that guarantees that synchronisation will be made for 8 bytes or
2 instructions starting from the address requested, although if cache
lines are wider then a larger area may be synchronised.

Change the call to point to the first of the two instructions aka `emul'
instead, removing unpredictable behaviour resulting from cache
incoherency.

This bug only ever manifested itself on systems implementing 4-byte
cache lines, typically MIPS I systems, causing all kinds of weirdness.
This is because the sequence of two instructions starting from `emul' is
8-byte aligned and for 8-byte or wider cache lines the line synchronised
will span both, so the vast majority of systems have escaped unharmed.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9698/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The `ieee754sp_isnan' and `ieee754dp_isnan' NaN classifiers are now no
longer externally referred, remove their header prototypes and make them
local to the two only respective places still making use of them.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9693/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Rewrite qNaN handling in `ieee754sp_fdp' using the `ieee754_class_nan'
helper recently added, removing the external call to `ieee754sp_isnan'
and reducing the size of code by 16 instructions or 64 bytes.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9692/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

None of the comparison helpers in ieee754.h is used, remove them.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9691/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Remove a redundant call to `ieee754_setandtestcx' in `ieee754sp_cmp' and
`ieee754dp_cmp'.  The IEEE 754 exception requested will have already
been set by a call to `ieee754_setcx' immediately above, because `sig'
has to be non-zero to reach here, and the comparison result returned
will be 0 regardless of the result from the call.  Simplify the return
expression remaining.  All this reducing the size of code by 16 and 12
instructions or 64 and 48 bytes respectively.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9690/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

We have the input operands already classified in `ieee754sp_cmp' and
`ieee754dp_cmp' comparison operations, so use the class obtained to tell
NaNs and numbers apart rather than classifying inputs again for this
purpose, reducing the size of code by 24 and 40 instructions or 96 and
160 bytes respectively.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9689/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Revert the changes made by commit fdffbafb [Lots of FPU bug fixes from
Kjeld Borch Egevang.] to `ieee754sp_nanxcpt' and `ieee754dp_nanxcpt'
sNaN quieting handlers and their callers so that sNaN processing is done
within the handlers againg.  Pass the sNaN causing an IEEE 754 invalid
operation exception down to the relevant handler.  Pass the sNaN in `fs'
where two sNaNs are supplied to a binary operation.

Set the Invalid Operation FCSR exception bits in the quieting handlers
rather than at their call sites throughout.  Make the handlers exclusive
for sNaN processing.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9688/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Don't call the `ieee754sp_nanxcpt' and `ieee754dp_nanxcpt' sNaN quieting
handlers for a qNaN supplied to floating-point format conversions or
SQRT.S/SQRT.D instructions, or for a qNaN produced out of a negative
operand supplied to SQRT.S/SQRT.D instructions.  Return the qNaN right
away in these cases.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9687/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9686/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Commit fdffbafb [Lots of FPU bug fixes from Kjeld Borch Egevang.]
replaced the two single `ieee754sp_nanxcpt' and `ieee754dp_nanxcpt'
places, where sNaN quieting used to happen for single and double
floating-point operations respectively, with individual qNaN
instantiations across all the call sites instead.  It also made most of
these two functions dead code as where called on a qNaN they return
right away.

To revert the damage and make sNaN quieting uniform again first rewrite
`ieee754sp_nanxcpt' and `ieee754dp_nanxcpt' to do the same quieting all
the call sites do, that is return the default qNaN encoding for all
input sNaN values; never propagate any sNaN payload bits from its
trailing significand field.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9685/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Move CFC1/CTC1 emulation code to separate functions to avoid excessive
indentation in forthcoming changes.  Adjust formatting in a minor way
and remove extraneous round brackets.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9682/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Commit 56a64733 [MIPS: math-emu: Switch to using the MIPS rounding
modes.] removed the distinction between hardware and emulator rounding
mode encodings, the hardware encoding is now used in emulation as well.
Complement the change and remove the `modeindex' macro previously used
for indexing into encoding translation tables, it now does nothing and
only obfuscates code by reinserting the value extracted from FCSR.
Adjust comments accordingly.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9680/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9676/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Correct the double-tab indentation of the branch-likely not-taken case.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9674/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Add the remaining missing comments for IEEE 754 special value array
indices.  Reindent macro definitions for consistency.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9671/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

IEEE754_SPCVAL_NMIN denotes the index into the special value array where
the closest to zero negative normal number expressible is stored.
Similarly IEEE754_SPCVAL_NMIND denotes such index for the closest to
zero negative subnormal number expressible.  Make comments match that.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9670/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Delay slot emulation in the FPU emulator is the only kernel user of an
executable stack, it is also very slow.  Add a counter so we can see
how many of these emulations are done.
Signed-off-by: NDavid Daney <david.daney@cavium.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/8634/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Remove variable self-assignments.
This silences a bunch of -Wself-assign warnings reported by clang.
The changed code can be compiled without warnings by both gcc and clang.
Signed-off-by: NToma Tabacu <toma.tabacu@imgtec.com>
Signed-off-by: NDaniel Sanders <daniel.sanders@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/9314/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

MIPS R2 FPU instructions are also present in MIPS R6 so amend the
preprocessor definitions to take MIPS R6 into consideration.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

MIPS R6 removed quite a few R2 instructions. However, there
is plenty of <R6 userland code so we add an in-kernel emulator
so we can still be able to execute all R2 userland out there.

The emulator comes with a handy debugfs under /mips/ directory
(r2-emul-stats) to provide some basic statistics of the
instructions that are being emulated.

Below are some statistics from booting a minimal buildroot image:

Instruction     Total   BDslot
------------------------------
movs            236969  0
hilo            56686   0
muls            55279   0
divs            10941   0
dsps            0       0
bops            1       0
traps           0       0
fpus            0       0
loads           214981  17
stores          103364  0
llsc            56898   0
dsemul          150418  0
jr              370158
bltzl           43
bgezl           1594
bltzll          0
bgezll          0
bltzal          39
bgezal          39
beql            14503
bnel            138741
blezl           0
bgtzl           3988
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

MIPS R6 uses the <R6 sdc2 opcode for the new BNEZC and JIALC instructions
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

MIPS R6 uses the <R6 ldc2 opcode for the new BEQZC and JIC instructions
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

MIPS R6 uses the <R6 swc2 opcode for the new BALC instructions.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

MIPS R6 uses the <R6 DADDI opcode for the new BNVC, BNEC and
BNEZLAC instructions.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

MIPS R6 uses the <R6 ADDI opcode for the new BOVC, BEQC and
BEQZALC instructions.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>