Modify the functions we use to read information about the topology of
the system (the number of cores, VPs & IOCUs that it contains) in order
to take into account multiple clusters, and provide a new function to
determine the number of clusters in the system.

Users of these functions are modified only such that they continue to
build successfully - having them actually handle multiple clusters is
left to further patches.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17016/
Patchwork: https://patchwork.linux-mips.org/patch/17218/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

With Coherence Manager (CM) 3.5 information about the topology of the
system, which has previously only been available through & accessed from
the CM, is now also provided by the Cluster Power Controller (CPC). This
includes a new CPC_CONFIG register mirroring GCR_CONFIG, and similarly a
new CPC_Cx_CONFIG register mirroring GCR_Cx_CONFIG.

In preparation for adjusting functions such as mips_cm_numcores(), which
have previously only needed to access the CM, to also access the CPC
this patch modifies the way we use the various CPS headers. Rather than
having users include asm/mips-cm.h or asm/mips-cpc.h individually we
instead have users include asm/mips-cps.h which in turn includes
asm/mips-cm.h & asm/mips-cpc.h. This means that users will gain access
to both CM & CPC registers by including one header, and most importantly
it makes asm/mips-cps.h an ideal location for helper functions which
need to access the various components of the CPS.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17015/
Patchwork: https://patchwork.linux-mips.org/patch/17217/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Allow the boot_secondary SMP op to return an error to __cpu_up(), which
will in turn return it to its caller.

This will allow SMP implementations to return errors quickly in cases
they they know have failed, rather than relying upon __cpu_up()
eventually timing out waiting for the cpu_running completion.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17014/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

With CM >= 3.5 we have the notion of multiple clusters & can access
their CM, CPC & GIC registers via the apporpriate redirect/other
register blocks. In order to allow for this introduce cluster & block
arguments to mips_cm_lock_other() which configures the redirect/other
region to point at the appropriate cluster, core, VP & register block.

Since we now have 4 arguments to mips_cm_lock_other() & a common use is
likely to be to target the cluster, core & VP corresponding to a
particular Linux CPU number we also add a new mips_cm_lock_other_cpu()
helper function which handles that without the caller needing to
manually pull out the cluster, core & VP numbers.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17013/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Introduce cpu_cluster() & cpu_set_cluster() accessor functions in the
same vein as cpu_core(), cpu_vpe_id() & their set variants. These will
be used in further patches to allow users to get or set a CPUs cluster
number.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17012/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Up until now we have open-coded checks for whether CPUs are siblings,
with slight variations on whether we consider the package ID or not.

This will only get more complex when we introduce cluster support, so in
preparation for that this patch introduces a cpus_are_siblings()
function which can be used to check whether or not 2 CPUs are siblings
in a consistent manner.

By checking globalnumber with the VP ID masked out this also has the
neat side effect of being ready for multi-cluster systems already.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Acked-by: NRafael J. Wysocki <rjw@rjwysocki.net>
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17011/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

This patch modifies the way we store core & VP IDs such that we store
them in a single 32 bit integer whose format matches that of the MIPSr6
GlobalNumber register. Whereas we have previously stored core & VP IDs
in separate fields, storing them in a single GlobalNumber-like field:

  1) Reduces the size of struct cpuinfo_mips by 4 bytes, and will allow
     it to not grow when cluster support is added.

  2) Gives us a natural place to store cluster number, which matches up
     with what the architecture provides.

  3) Will be useful in the future as a parameter to the MIPSr6 GINVI
     instruction to specify a target CPU whose icache that instruction
     should operate on.

The cpu_set*() accessor functions are moved out of the asm/cpu-info.h
header in order to allow them to use the WARN_ON macro, which is
unusable in asm/cpu-info.h due to include ordering.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17010/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

We currently have fields in struct cpuinfo_mips for the core & VP(E) ID
of a particular CPU, and various pieces of code directly access those
fields. This patch abstracts such access by introducing accessor
functions cpu_core(), cpu_set_core(), cpu_vpe_id() & cpu_set_vpe_id()
and having code that needs to access these values call those functions
rather than directly accessing the struct cpuinfo_mips fields. This
prepares us for changes to the way in which those values are stored in
later patches.

The cpu_vpe_id() function is introduced even though we already had a
cpu_vpe_id() macro for a couple of reasons:

  1) It's more consistent with the core, and future cluster, accessors.

  2) It ensures a sensible return type without explicit casts.

  3) It's generally preferable to use functions rather than macros.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17009/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

We now have definitions for the GlobalNumber register in asm/mipsregs.h,
so use them in place of magic numbers in cps-vec.S.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17008/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

MIPSr6 introduces a GlobalNumber register, which is required when VPs
are implemented (ie. when multi-threading is supported) but otherwise
optional. The register contains sufficient information to uniquely
identify a VP within a system using its cluster number, core number & VP
ID.

In preparation for using this register & its fields, introduce an
accessor macro for it & define its various bits with the typical style
preprocessor macros.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17007/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Introduce definitions & accessors for a selection of Coherence Manager
(CM) & Cluster Power Controller (CPC) registers that are new with CM
v3.5 & the MIPS I6500. These are primarily registers that will be used
in supporting multiple CPU clusters.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17006/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Make use of the new change_*, set_* & clear_* accessor functions for CPS
(CM, CPC & GIC) registers where doing so makes the code easier to read
or shortens it without adversely affecting readability.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17005/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

For read-write registers introduce accessor functions that simplify the
task of modifying a subset of bits within the register. set_* functions
set bits to 1, clear_* functions clear bits to 0 & change_* functions
set bits specified in a mask to an arbitrary value.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17004/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Tidy up asm/mips-cpc.h in a similar way to what "MIPS: CM: Use
BIT/GENMASK for register fields, order & drop shifts" did for
asm/mips-cm.h.

We use BIT() & GENMASK() to simplify the definition of register fields,
drop the _SHF definitions since that information can be found in the
_MSK ones, and then drop the _MSK suffix.

Fields definitions are moved to be next to the appropriate register
definition, making it easier to link the two & keep everything ordered
by register address. Comments are added including the name of each
register & a brief description of its purpose which helps to understand
what registers are for, link them back to hardware documentation or grep
for them.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17003/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Switch the MIPS Cluster Power Controller (CPC) accessor functions to be
generated by the new common Coherent Processing System (CPS) macros
shared with the Coherence Manager (CM).
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17002/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

There's no reason for us not to use BIT() & GENMASK() in asm/mips-cm.h
when declaring macros corresponding to register fields. This patch
modifies our definitions to do so.

The *_SHF definitions are removed entirely - they duplicate information
found in the masks, are infrequently used & can be replaced with use of
__ffs() where needed.

The *_MSK definitions then lose their _MSK suffix which is now somewhat
redundant, and users are modified to match.

The field definitions are moved to follow the appropriate register's
accessor functions, which helps to keep the field definitions in order &
to find the appropriate fields for a given register. Whilst here a
comment is added describing each register & including its name, which is
helpful both for linking the register back to hardware documentation &
for grepping purposes.

This also cleans up a couple of issues that became obvious as a result
of making the changes described above:

  - We previously had definitions for GCR_Cx_RESET_EXT_BASE & a phony
    copy of that named GCR_RESET_EXT_BASE - a register which does not
    exist. The bad definitions were added by commit 497e803e ("MIPS:
    smp-cps: Ensure secondary cores start with EVA disabled") and made
    use of from boot_core(), which is now modified to use the
    GCR_Cx_RESET_EXT_BASE definitions.

  - We had a typo in CM_GCR_ERROR_CAUSE_ERRINGO_MSK - we now correctly
    define this as inFo rather than inGo.

Now that we don't duplicate field information between _SHF & _MSK
definitions, and keep the fields next to the register accessors, it will
be much easier to spot & prevent any similar oddities being introduced
in the future.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Acked-by: Thomas Gleixner <tglx@linutronix.de
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17001/
Patchwork: https://patchwork.linux-mips.org/patch/17216/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Some CM registers are always 32 bits, or at least only use bits in the
lower 32 bits of the register. For these registers it is wasteful for us
to generate accessors which bother to check mips_cm_is64 & perform 64
bit accesses.

This patch modifies the accessor generation to take into account the
size of the register, and for 32 bit registers we generate accessors
which only ever perform 32 bit accesses. For 64 bit registers we either
perform a 64 bit access or two 32 bit accesses, depending upon the value
of mips_cm_is64. Doing this saves us ~1.5KiB of code in a generic 64r6el
kernel, and perhaps more importantly simplifies various code paths.

This removes the read64_gcr_* accessors, so mips_cm_error_report() is
modified to stop using them & instead use the regular read_gcr_*
accessors which will return 64 bit values from the 64 bit registers.

The new accessor macros are placed in asm/mips-cps.h such that they can
be shared by CPC & GIC code in later patches.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/17000/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

We currently have a mips_cm_base variable which holds the base address
of the Coherence Manager (CM) Global Configuration Registers (GCRs), and
accessor functions which use the GCR in their names. This works fine,
but gets in the way of sharing the code to generate the accessor
functions with other blocks (ie. CPC & GIC) because that code would then
need to separately handle the name of the base address variable & the
name used in the accessor functions.

In order to prepare for sharing the accessor generation code between CM,
CPC & GIC code this patch renames mips_cm_base to mips_gcr_base such
that the "gcr" portion is common to both the base address variable & the
accessor function names.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/16999/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Add FP emulation debugfs statistics for individual instructions. The
debugfs files that contain counter values are placed in a separate
directory called "instructions". This means that the default path for
these new stat is "/sys/kernel/debug/mips/fpuemustats/instructions".

Each instruction counter is mapped to the debugfs file that has the
same name as instruction name. The lowercase is choosen as more
commonly used case for instruction names.

One example of usage:

mips_host::/sys/kernel/debug/mips/fpuemustats/instructions # grep "" *

The shortened output of this command is:

abs.d:34
abs.s:5711
add.d:10401
add.s:399307
bc1eqz:3199
...
...
...
sub.s:167211
trunc.l.d:375
trunc.l.s:8054
trunc.w.d:421
trunc.w.s:27032

The limitation of this patch is that it handles R6 FP emulation
instructions only. There are altogether 114 handled instructions.
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Maciej W. Rozycki <macro@imgtec.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/17145/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Add capability for the user to clear all FP emu debugfs counters.

This is achieved by having a special debugfs file "fpuemustats_clear"
(under default location "/sys/kernel/debug/mips"). Each access to the
file results in setting all counters to zero (it is enough, let's say,
to issue a "cat /sys/kernel/debug/mips/fpuemustats_clear").

This functionality already exists for R2 emulation statistics,
but was missing for FP emulation statistics. The implementation in
this patch is consistent with its R2 emulation counterpart.
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/17144/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Add FP emu debugfs counter for branches.

The new counter is displayed the same way as existing counter, and
its default path is /sys/kernel/debug/mips/fpuemustats/.

The limitation of this counter is that it counts only R6 branch
instructions BC1NEZ and BC1EQZ.
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Maciej W. Rozycki <macro@imgtec.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/17143/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix content of CLASS.D output bits 32-63 to match hardware behavior.

Prior to this patch, bits 32-63 of CLASS.D output were not
initialized, causing different 32-63 bits content of CLASS.D, based on
circumstances. However, the hardware consistently returns all these
bits zeroed. The documentation is not clear whether these bits should
be zero or unpredictable. Since technically "all zero" case still can
be viewed as belonging to "unpredictable" class of results, it is
better to zero bits 32-63.
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Maciej W. Rozycki <macro@imgtec.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/17142/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Reimplement RINT.<D|S> kernel emulation so that all RINT.<D|S>
specifications are met.

For the sake of simplicity, let's analyze RINT.S only. Prior to
this patch, RINT.S emulation was essentially implemented as (in
pseudocode) <output> = ieee754sp_flong(ieee754sp_tlong(<input>)),
where ieee754sp_tlong() and ieee754sp_flong() are functions
providing conversion from double to integer, and from integer
to double, respectively. On surface, this implementation looks
correct, but actually fails in many cases. Following problems
were detected:

1. NaN and infinity cases will not be handled properly. The
   function ieee754sp_flong() never returns NaN nor infinity.
2. For RINT.S, for all inputs larger than LONG_MAX, and smaller
   than FLT_MAX, the result will be wrong, and the overflow
   exception will be erroneously set. A similar problem for
   negative inputs exists as well.
3. For some rounding modes, for some negative inputs close to zero,
   the return value will be zero, and should be -zero. This is
   because ieee754sp_flong() never returns -zero.

This patch removes the problems above by implementing dedicated
functions for RINT.<D|S> emulation.

The core of the new function functionality is adapted version of
the core of the function ieee754sp_tlong(). However, there are many
details that are implemented to match RINT.<D|S> specification. It
should be said that the functionality of ieee754sp_tlong() actually
closely corresponds to CVT.L.S instruction, and it is used while
emulating CVT.L.S. However, RINT.S and CVT.L.S instructions differ
in many aspects. This patch fulfills missing support for RINT.<D|S>.
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hans Verkuil <hans.verkuil@cisco.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Maciej W. Rozycki <macro@imgtec.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/17141/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix CMP.Sxxx.<D|S> SIGILL crashes by fixing main switch/case statement
in fpu_emul() function so that inadvertent fall-troughs are prevented.

Consider, let's say, CMP.SAF.S instruction when one of inputs is zero
and another input is a signaling NaN. The desired output is zero, and
the exception flag "invalid operation" set. For such case, the main
portion of the implementation is within "d_fmt" case of the main
"switch/case" statement in fpu_emul() function. The execution will
follow one of "if-else" branches that doesn't contain "goto cop1scr;"
statement, and will therefore reach the end of "d_fmt" case. It will
subsequently fall through to the next case, "l_fmt". After following
similar pattern, the execution will fall through to the succeeding
case, which is "default". The "default" case contains "return SIGILL;"
statement only. This means that the caller application will crash
with "illegal instruction" message.

It is obvious that above described fall-throughs are unnecessary and
harmful. This patch rectifies that behavior by providing "break;"
statements at the end of cases "d_fmt" and "l_fmt".

There are 22 instructions affected by this problem:

CMP.<SAF|SEQ|SLE|SLT|SNE|SOR|SUEQ|SULE|SULT|SUN|SUNE>.<D|S>.
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Maciej W. Rozycki <macro@imgtec.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/17140/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Implement fused multiply-add with correct accuracy.

Fused multiply-add operation has better accuracy than respective
sequential execution of multiply and add operations applied on the
same inputs. This is because accuracy errors accumulate in latter
case.

This patch implements fused multiply-add with the same accuracy
as it is implemented in hardware, using 128-bit intermediate
calculations.

One test case example (raw bits) that this patch fixes:

MADDF.D fd,fs,ft:
  fd = 0x00000ca000000000
  fs = ft = 0x3f40624dd2f1a9fc

Fixes: e24c3bec ("MIPS: math-emu: Add support for the MIPS R6 MADDF FPU instruction")
Fixes: 83d43305 ("MIPS: math-emu: Add support for the MIPS R6 MSUBF FPU instruction")
Signed-off-by: NDouglas Leung <douglas.leung@imgtec.com>
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Bo Hu <bohu@google.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Jin Qian <jinqian@google.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.7+
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16891/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Implement fused multiply-add with correct accuracy.

Fused multiply-add operation has better accuracy than respective
sequential execution of multiply and add operations applied on the
same inputs. This is because accuracy errors accumulate in latter
case.

This patch implements fused multiply-add with the same accuracy
as it is implemented in hardware, using 64-bit intermediate
calculations.

One test case example (raw bits) that this patch fixes:

MADDF.S fd,fs,ft:
  fd = 0x22575225
  fs = ft = 0x3727c5ac

Fixes: e24c3bec ("MIPS: math-emu: Add support for the MIPS R6 MADDF FPU instruction")
Fixes: 83d43305 ("MIPS: math-emu: Add support for the MIPS R6 MSUBF FPU instruction")
Signed-off-by: NDouglas Leung <douglas.leung@imgtec.com>
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Bo Hu <bohu@google.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Jin Qian <jinqian@google.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.7+
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16890/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix definition and usage of "maddf_flags" enumeration. Avoid duplicate
definition and apply more common capitalization.

This patch does not change any scenario. It just makes MADDF and
MSUBF emulation code more readable and easier to maintain, and
hopefully prevents future bugs as well.
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Reviewed-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Bo Hu <bohu@google.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Jin Qian <jinqian@google.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.7+
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16889/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix the cases of <MADDF|MSUBF>.<D|S> when any of two multiplicands is
+0 or -0, and the third input is also +0 or -0. Depending on the signs
of inputs, certain special cases must be handled.

A relevant example:

MADDF.S fd,fs,ft:
  If fs contains +0.0, ft contains -0.0, and fd contains 0.0, fd is
  going to contain +0.0 (without this patch, it used to contain -0.0).

Fixes: e24c3bec ("MIPS: math-emu: Add support for the MIPS R6 MADDF FPU instruction")
Fixes: 83d43305 ("MIPS: math-emu: Add support for the MIPS R6 MSUBF FPU instruction")
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Reviewed-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Bo Hu <bohu@google.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Jin Qian <jinqian@google.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.7+
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16888/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix the cases of <MADDF|MSUBF>.<D|S> when any of two multiplicands is
infinity. The correct behavior in such cases is affected by the nature
of third input. Cases of addition of infinities with opposite signs
and subtraction of infinities with same signs may arise and must be
handles separately. Also, the value od flags argument (that determines
whether the instruction is MADDF or MSUBF) affects the outcome.

Relevant examples:

MADDF.S fd,fs,ft:
  If fs contains +inf, ft contains +inf, and fd contains -inf, fd is
  going to contain indef (without this patch, it used to contain
  -inf).

MSUBF.S fd,fs,ft:
  If fs contains +inf, ft contains 1.0, and fd contains +0.0, fd is
  going to contain -inf (without this patch, it used to contain +inf).

Fixes: e24c3bec ("MIPS: math-emu: Add support for the MIPS R6 MADDF FPU instruction")
Fixes: 83d43305 ("MIPS: math-emu: Add support for the MIPS R6 MSUBF FPU instruction")
Signed-off-by: NDouglas Leung <douglas.leung@imgtec.com>
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Reviewed-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Bo Hu <bohu@google.com>
Cc: Jin Qian <jinqian@google.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.7+
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16887/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix the cases of <MADDF|MSUBF>.<D|S> when any of three inputs is any
NaN. Correct behavior of <MADDF|MSUBF>.<D|S> fd, fs, ft is following:

  - if any of inputs is sNaN, return a sNaN using following rules: if
    only one input is sNaN, return that one; if more than one input is
    sNaN, order of precedence for return value is fd, fs, ft
  - if no input is sNaN, but at least one of inputs is qNaN, return a
    qNaN using following rules: if only one input is qNaN, return that
    one; if more than one input is qNaN, order of precedence for
    return value is fd, fs, ft

The previous code contained correct handling of some above cases, but
not all. Also, such handling was scattered into various cases of
"switch (CLPAIR(xc, yc))" statement, and elsewhere. With this patch,
this logic is placed in one place, and "switch (CLPAIR(xc, yc))" is
significantly simplified.

A relevant example:

MADDF.S fd,fs,ft:
  If fs contains qNaN1, ft contains qNaN2, and fd contains qNaN3, fd
  is going to contain qNaN3 (without this patch, it used to contain
  qNaN1).

Fixes: e24c3bec ("MIPS: math-emu: Add support for the MIPS R6 MADDF FPU instruction")
Fixes: 83d43305 ("MIPS: math-emu: Add support for the MIPS R6 MSUBF FPU instruction")
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Reviewed-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Bo Hu <bohu@google.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Jin Qian <jinqian@google.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.7+
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16886/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix following special cases for MINA>.<D|S>:

  - if one of the inputs is zero, and the other is subnormal, normal,
    or infinity, the  value of the former should be returned (that is,
    a zero).
  - if one of the inputs is infinity, and the other input is normal,
    or subnormal, the value of the latter should be returned.

The previous implementation's logic for such cases was incorrect - it
appears as if it implements MAXA, and not MINA instruction.

A relevant example:

MINA.S fd,fs,ft:
  If fs contains 100.0, and ft contains 0.0, fd is going to contain
  0.0 (without this patch, it used to contain 100.0).

Fixes: a79f5f9b ("MIPS: math-emu: Add support for the MIPS R6 MAX{, A} FPU instruction")
Fixes: 4e9561b2 ("MIPS: math-emu: Add support for the MIPS R6 MIN{, A} FPU instruction")
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Reviewed-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Bo Hu <bohu@google.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Jin Qian <jinqian@google.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.3+
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16885/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix the value returned by <MAXA|MINA>.<D|S> fd,fs,ft, if both inputs
are infinite. The previous implementation returned always the value
contained in ft in such cases. The correct behavior is specified
in Mips instruction set manual and is as follows:

    fs    ft        MAXA     MINA
  ---------------------------------
    inf   inf        inf      inf
    inf  -inf        inf     -inf
   -inf   inf        inf     -inf
   -inf  -inf       -inf     -inf

A relevant example:

MAXA.S fd,fs,ft:
  If fs contains +inf, and ft contains -inf, fd is going to contain
  +inf (without this patch, it used to contain -inf).

Fixes: a79f5f9b ("MIPS: math-emu: Add support for the MIPS R6 MAX{, A} FPU instruction")
Fixes: 4e9561b2 ("MIPS: math-emu: Add support for the MIPS R6 MIN{, A} FPU instruction")
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Reviewed-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Bo Hu <bohu@google.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Jin Qian <jinqian@google.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.3+
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16884/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix the value returned by <MAXA|MINA>.<D|S>, if the inputs are normal
fp numbers of the same absolute value, but opposite signs.

A relevant example:

MAXA.S fd,fs,ft:
  If fs contains -3.0, and ft contains +3.0, fd is going to contain
  +3.0 (without this patch, it used to contain -3.0).

Fixes: a79f5f9b ("MIPS: math-emu: Add support for the MIPS R6 MAX{, A} FPU instruction")
Fixes: 4e9561b2 ("MIPS: math-emu: Add support for the MIPS R6 MIN{, A} FPU instruction")
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Reviewed-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Bo Hu <bohu@google.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Jin Qian <jinqian@google.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.3+
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16883/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix the value returned by <MAX|MIN>.<D|S>, if both inputs are negative
normal fp numbers. The previous logic did not take into account that
if both inputs have the same sign, there should be separate treatment
of the cases when both inputs are negative and when both inputs are
positive.

A relevant example:

MAX.S fd,fs,ft:
  If fs contains -5.0, and ft contains -7.0, fd is going to contain
  -5.0 (without this patch, it used to contain -7.0).

Fixes: a79f5f9b ("MIPS: math-emu: Add support for the MIPS R6 MAX{, A} FPU instruction")
Fixes: 4e9561b2 ("MIPS: math-emu: Add support for the MIPS R6 MIN{, A} FPU instruction")
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Reviewed-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Bo Hu <bohu@google.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Jin Qian <jinqian@google.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.3+
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16882/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix the value returned by <MAX|MAXA|MIN|MINA>.<D|S>, if both inputs
are zeros. The right behavior in such cases is stated in instruction
reference manual and is as follows:

   fs  ft       MAX     MIN       MAXA    MINA
  ---------------------------------------------
    0   0        0       0         0       0
    0  -0        0      -0         0      -0
   -0   0        0      -0         0      -0
   -0  -0       -0      -0        -0      -0

Prior to this patch, some of the above cases were yielding correct
results. However, for the sake of code consistency, all such cases
are rewritten in this patch.

A relevant example:

MAX.S fd,fs,ft:
  If fs contains +0.0, and ft contains -0.0, fd is going to contain
  +0.0 (without this patch, it used to contain -0.0).

Fixes: a79f5f9b ("MIPS: math-emu: Add support for the MIPS R6 MAX{, A} FPU instruction")
Fixes: 4e9561b2 ("MIPS: math-emu: Add support for the MIPS R6 MIN{, A} FPU instruction")
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Reviewed-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Bo Hu <bohu@google.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Jin Qian <jinqian@google.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.3+
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16881/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Fix the value returned by <MAX|MAXA|MIN|MINA>.<D|S> fd,fs,ft, if both
inputs are quiet NaNs. The <MAX|MAXA|MIN|MINA>.<D|S> specifications
state that the returned value in such cases should be the quiet NaN
contained in register fs.

A relevant example:

MAX.S fd,fs,ft:
  If fs contains qNaN1, and ft contains qNaN2, fd is going to contain
  qNaN1 (without this patch, it used to contain qNaN2).

Fixes: a79f5f9b ("MIPS: math-emu: Add support for the MIPS R6 MAX{, A} FPU instruction")
Fixes: 4e9561b2 ("MIPS: math-emu: Add support for the MIPS R6 MIN{, A} FPU instruction")
Signed-off-by: NMiodrag Dinic <miodrag.dinic@imgtec.com>
Signed-off-by: NGoran Ferenc <goran.ferenc@imgtec.com>
Signed-off-by: NAleksandar Markovic <aleksandar.markovic@imgtec.com>
Reviewed-by: NJames Hogan <james.hogan@imgtec.com>
Cc: Bo Hu <bohu@google.com>
Cc: Douglas Leung <douglas.leung@imgtec.com>
Cc: Jin Qian <jinqian@google.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Petar Jovanovic <petar.jovanovic@imgtec.com>
Cc: Raghu Gandham <raghu.gandham@imgtec.com>
Cc: <stable@vger.kernel.org> # 4.3+
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/16880/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

We currently have various variables & functions which are only used
within a single translation unit, but which we don't declare static.
This causes various sparse warnings of the form:

  arch/mips/kernel/mips-r2-to-r6-emul.c:49:1: warning: symbol
    'mipsr2emustats' was not declared. Should it be static?

  arch/mips/kernel/unaligned.c:1381:11: warning: symbol 'reg16to32st'
    was not declared. Should it be static?

  arch/mips/mm/mmap.c:146:15: warning: symbol 'arch_mmap_rnd' was not
    declared. Should it be static?

Fix these & others by declaring various affected variables & functions
static, avoiding the sparse warnings & redundant symbols.

[ralf@linux-mips.org: Add Marcin's build fix.]
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: trivial@kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/17176/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The plat_timer_setup() function is entirely unused - nothing calls it,
and no platforms provide it. Perhaps our dummy implementation was once
useful as an aid in forward porting platforms, but its time has long
since passed so let's remove the dead code.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: trivial@kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/17175/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The __invalidate_kernel_vmap_range function pointer global variable
isn't used anywhere. Remove it.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: trivial@kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/17174/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The fault_addr argument to fpu_emulator_cop1Handler(), fpux_emu() and
cop1Emulate() has up until now been declared as:

  void *__user *fault_addr

This is essentially a pointer in user memory which points to a pointer
to void. This is not the intent for our code, which is actually
operating on a pointer to a pointer to void where the pointer to void is
pointing at user memory. ie. the pointer is in kernel memory & points to
user memory.

This mismatch produces a lot of sparse warnings that look like this:

arch/mips/math-emu/cp1emu.c:1485:45:
   warning: incorrect type in assignment (different address spaces)
      expected void *[noderef] <asn:1><noident>
      got unsigned int [noderef] [usertype] <asn:1>*[assigned] va

Fix these by modifying the declaration of the fault_addr argument to:

  void __user **fault_addr
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: trivial@kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/17173/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>