The R12000 added a new feature to enhance branch prediction called
"global history".  Per the Vr10000 Series User Manual (U10278EJ4V0UM),
Coprocessor 0, Diagnostic Register (22):

"""
If bit 26 is set, branch prediction uses all eight bits of the global
history register.  If bit 26 is not set, then bits 25:23 specify a count
of the number of bits of global history to be used. Thus if bits 26:23
are all zero, global history is disabled.

The global history contains a record of the taken/not-taken status of
recently executed branches, and when used is XOR'ed with the PC of a
branch being predicted to produce a hashed value for indexing the BPT.
Some programs with small "working set of conditional branches" benefit
significantly from the use of such hashing, some see slight performance
degradation.
"""

This patch enables global history on R12000 CPUs and up by setting bit
26 in the branch prediction diagnostic register (CP0 $22) to '1'.  Bits
25:23 are left alone so that all eight bits of the global history
register are available for branch prediction.
Signed-off-by: NJoshua Kinard <kumba@gentoo.org>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Returns a non-zero value if the current processor implementation requires
an IHB instruction to deal with an instruction hazard as per MIPS R2
architecture specification, zero otherwise.

For a discussion, see http://patchwork.linux-mips.org/patch/9539/.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Correct ISA requirements for floating-point instructions:

* the CU3 exception signifies a real COP3 instruction in MIPS I & II,

* the BC1FL and BC1TL instructions are not supported in MIPS I,

* the SQRT.fmt instructions are indeed supported in MIPS II,

* the LDC1 and SDC1 instructions are indeed supported in MIPS32r1,

* the CEIL.W.fmt, FLOOR.W.fmt, ROUND.W.fmt and TRUNC.W.fmt instructions
  are indeed supported in MIPS32,

* the CVT.L.fmt and CVT.fmt.L instructions are indeed supported in
  MIPS32r2 and MIPS32r6,

* the CEIL.L.fmt, FLOOR.L.fmt, ROUND.L.fmt and TRUNC.L.fmt instructions
  are indeed supported in MIPS32r2 and MIPS32r6,

* the RSQRT.fmt and RECIP.fmt instructions are indeed supported in
  MIPS64r1,

Also simplify conditionals for MIPS III and MIPS IV FPU instructions and
the handling of the MOVCI minor opcode.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9700/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The `cpu_has_fpu' feature flag must not be hardcoded to 1 or the `nofpu'
kernel option will be ignored.  Remove any such overrides and add a
cautionary note.  Hardcoding to 0 is fine for FPU-less platforms.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9694/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Add architectural definitions and probing for the MIPS Common Device
Memory Map (CDMM) region. When supported and enabled at a particular
physical address, this region allows some number of per-CPU devices to
be discovered and controlled via MMIO.

A bit exists in Config3 to determine whether the feature is present, and
a CDMMBase CP0 register allows the region to be enabled at a particular
physical address.

[ralf@linux-mips.org: Sort conflict with other patches.]
Signed-off-by: NJames Hogan <james.hogan@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/9178/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Add support for extended physical addressing (XPA) so that
32-bit platforms can access equal to or greater than 40 bits
of physical addresses.

NOTE:
      1) XPA and EVA are not the same and cannot be used
         simultaneously.
      2) If you configure your kernel for XPA, the PTEs
         and all address sizes become 64-bit.
      3) Your platform MUST have working HIGHMEM support.
Signed-off-by: NSteven J. Hill <Steven.Hill@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/9355/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

MIPS R2 FPU instructions are also present in MIPS R6 so amend the
preprocessor definitions to take MIPS R6 into consideration.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

The LLBIT (bit 4) in the Config5 CP0 register indicates the software
availability of the Load-Linked bit. This bit is only set by hardware
and it has the following meaning:

0: LLB functionality is not supported
1: LLB functionality is supported. The following feature are also
supported:

- ERETNC instruction. Similar to ERET but it does not clear the LLB
bit in the LLAddr register.
- CP0 LLAddr/LLB bit must be set
- LLbit is software accessible through the LLAddr[0]

This will be used later on to emulate R2 LL/SC instructions.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Print 'mips64r6' and/or 'mips32r6' if the kernel is running on
a MIPS R6 core.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Add MIPS R6 to the ISA definitions
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

Detect the presence of the Config5 FRE & UFE bits, as indicated by the
FREP bit in FPIR. Record this as a CPU option bit, and provide a
cpu_has_fre macro to ease checking of that option bit.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: linux-fsdevel@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/7678/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: Nchenj <chenj@lemote.com>
Cc: linux-mips@linux-mips.org
Cc: chenhc@lemote.com
Patchwork: https://patchwork.linux-mips.org/patch/7542/
Patchwork: https://patchwork.linux-mips.org/patch/7550/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Detect the presence of MAAR using the MRP bit in Config5, and record
that presence using a CPU option bit. A cpu_has_maar macro will then
allow code to conditionalise upon the presence of MAARs.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/7330/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

MIPSr5 added support for unique exception codes for the Read-Inhibit
and Execute-Inhibit exceptions.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/7338/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Moreover, report hardware page table walker support as 'htw' in the ASE
list of /proc/cpuinfo, if the core implements this feature.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/7334/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The TLB handlers cannot handle this case, so disable it for now.
Signed-off-by: NDavid Daney <david.daney@cavium.com>
Signed-off-by: NAndreas Herrmann <andreas.herrmann@caviumnetworks.com>
Cc: linux-mips@linux-mips.org
Cc: James Hogan <james.hogan@imgtec.com>
Cc: kvm@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/7007/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Most of these tests should be runtime tests.  This also finally means
that on a MIPS III systems MIPS IV opcodes are going to result in an
exception as they're supposed to.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The MIPS *Aptiv family uses bit 28 in Config5 CP0 register to
indicate whether the core supports EVA or not.
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>

This patch adds support for probing the MSAP bit within the Config3
register in order to detect the presence of the MSA ASE. Presence of the
ASE will be indicated in /proc/cpuinfo. The value of the MSA
implementation register will be displayed at boot to aid debugging and
verification of a correct setup, as is done for the FPU.
Signed-off-by: NPaul Burton <paul.burton@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/6430/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

MIPS32R3 introduced a new set of Segmentation Control registers which
increase the flexibility of the segmented-based memory scheme.
Signed-off-by: NSteven J. Hill <Steven.Hill@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>
Signed-off-by: NJohn Crispin <blogic@openwrt.org>
Patchwork: http://patchwork.linux-mips.org/patch/6131/

New Aptiv cores support the TLBINVF instruction for flushing
the VTLB.
Signed-off-by: NLeonid Yegoshin <Leonid.Yegoshin@imgtec.com>
Signed-off-by: NMarkos Chandras <markos.chandras@imgtec.com>
Signed-off-by: NJohn Crispin <blogic@openwrt.org>
Patchwork: http://patchwork.linux-mips.org/patch/6130/

No support for MIPS53 processors yet.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/5876/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

 o Move current_cpu_type() to a separate header file
 o #ifdefing on supported CPU types lets modern GCC know that certain
   code in callers may be discarded ideally turning current_cpu_type() into
   a function returning a constant.
 o Use current_cpu_type() rather than direct access to struct cpuinfo_mips.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
Cc: Steven J. Hill <Steven.Hill@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/5833/

current_cpu_type() is not preemption-safe.
If CONFIG_PREEMPT is enabled then mipsxx_reg_setup() can be called from preemptible state.
Added get_cpu()/put_cpu() pair to make it preemption-safe.

This was found while testing oprofile with CONFIG_DEBUG_PREEMPT enable.

/usr/zntestsuite # opcontrol --init
/usr/zntestsuite # opcontrol --setup --event=L2_CACHE_ACCESSES:500 --event=L2_CACHE_MISSES:500 --no-vmlinux
/usr/zntestsuite # opcontrol --start
Using 2.6+ OProfile kernel interface.
BUG: using smp_processor_id() in preemptible [00000000] code: oprofiled/1362
caller is mipsxx_reg_setup+0x11c/0x164
CPU: 0 PID: 1362 Comm: oprofiled Not tainted 3.10.4 #18
Stack : 00000006 70757465 00000000 00000000 00000000 00000000 80b173f6 00000037
          80b10000 00000000 80b21614 88f5a220 00000000 00000000 00000000 00000000
          00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
          00000000 00000000 00000000 89c49c00 89c49c2c 80721254 807b7927 8012c1d0
          80b10000 80721254 00000000 00000552 88f5a220 80b1335c 807b78e6 89c49ba8
          ...
Call Trace:
[<801099a4>] show_stack+0x64/0x7c
[<80665520>] dump_stack+0x20/0x2c
[<803a2250>] debug_smp_processor_id+0xe0/0xf0
[<8052df24>] mipsxx_reg_setup+0x11c/0x164
[<8052cd70>] op_mips_setup+0x24/0x4c
[<80529cfc>] oprofile_setup+0x5c/0x12c
[<8052b9f8>] event_buffer_open+0x78/0xf8
[<801c3150>] do_dentry_open.isra.15+0x2b8/0x3b0
[<801c3270>] finish_open+0x28/0x4c
[<801d49b8>] do_last.isra.41+0x2cc/0xd00
[<801d54a0>] path_openat+0xb4/0x4c4
[<801d5c44>] do_filp_open+0x3c/0xac
[<801c4744>] do_sys_open+0x110/0x1f4
[<8010f47c>] stack_done+0x20/0x44

Bug reported and original patch by Jerin Jacob <jerinjacobk@gmail.com>.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
Acked-by: NJerin Jacob <jerinjacobk@gmail.com>

Signed-off-by: NTony Wu <tung7970@gmail.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/5536/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

As Jonas Gorske said in his patch:

   Disable cpu_has_mmips for everything but SEAD3 and MALTA. Most of
   these platforms are from before the micromips introduction, so they
   are very unlikely to implement it.

   Reduces an -Os compiled, uncompressed kernel image by 8KiB for
   BCM63XX.

This patch taks a different approach than his, we gate the runtime
test for microMIPS by the config symbol SYS_SUPPORTS_MICROMIPS.
Signed-off-by: NDavid Daney <david.daney@cavium.com>
Cc: Jonas Gorski <jogo@openwrt.org>
Cc: Steven J. Hill <Steven.Hill@imgtec.com>
Acked-by: NSteven J. Hill <Steven.Hill@imgtec.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/5327/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

MIPS I is the ancestor of all MIPS ISA and architecture variants.  Anything
ever build in the MIPS empire is either MIPS I or at least contains MIPS I.
If it's running Linux, that is.

So there is little point in having cpu_has_mips_1 because it will always
evaluate as true - though usually only at runtime.  Thus there is no
point in having the MIPS_CPU_ISA_I ISA flag, so get rid of it.

Little complication: traps.c was using a test for a pure MIPS I ISA as
a test for an R3000-style cp0.  To deal with that, use a check for
cpu_has_3kex or cpu_has_4kex instead.

cpu_has_3kex is a new macro.  At the moment its default implementation is
!cpu_has_4kex but this may eventually change if Linux is ever going to
support the oddball MIPS processors R6000 and R8000 so users of either
of these macros should not make any assumptions.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
Patchwork: https://patchwork.linux-mips.org/patch/5551/

This and the next patch resolve memory corruption problems while CPU
hotplug. Without these patches, memory corruption can triggered easily
as below:

On a quad-core MIPS platform, use "spawn" of UnixBench-5.1.3 (http://
code.google.com/p/byte-unixbench/) and a CPU hotplug script like this
(hotplug.sh):
while true; do
echo 0 >/sys/devices/system/cpu/cpu1/online
echo 0 >/sys/devices/system/cpu/cpu2/online
echo 0 >/sys/devices/system/cpu/cpu3/online
sleep 1
echo 1 >/sys/devices/system/cpu/cpu1/online
echo 1 >/sys/devices/system/cpu/cpu2/online
echo 1 >/sys/devices/system/cpu/cpu3/online
sleep 1
done

Run "hotplug.sh" and then run "spawn 10000", spawn will get segfault
after a few minutes.

This patch:
Currently, clear_page()/copy_page() are generated by Micro-assembler
dynamically. But they are unavailable until uasm_resolve_relocs() has
finished because jump labels are illegal before that. Since these
functions are shared by every CPU, we only call build_clear_page()/
build_copy_page() only once at boot time. Without this patch, programs
will get random memory corruption (segmentation fault, bus error, etc.)
while CPU Hotplug (e.g. one CPU is using clear_page() while another is
generating it in cpu_cache_init()).

For similar reasons we modify build_tlb_refill_handler()'s invocation.

V2:
1, Rework the code to make CPU#0 can be online/offline.
2, Introduce cpu_has_local_ebase feature since some types of MIPS CPU
   need a per-CPU tlb_refill_handler().
Signed-off-by: NHuacai Chen <chenhc@lemote.com>
Signed-off-by: NHongbing Hu <huhb@lemote.com>
Acked-by: NDavid Daney <david.daney@cavium.com>
Patchwork: http://patchwork.linux-mips.org/patch/4994/Acked-by: NJohn Crispin <blogic@openwrt.org>

The presence of the MIPS Virtualization Application-Specific Extension
is indicated by CP0_Config3[23].  Probe for this and report it in
/proc/cpuinfo.
Signed-off-by: NDavid Daney <david.daney@cavium.com>
Patchwork: http://patchwork.linux-mips.org/patch/4904/Signed-off-by: NJohn Crispin <blogic@openwrt.org>

Signed-off-by: NSteven J. Hill <sjhill@mips.com>
Patchwork: http://patchwork.linux-mips.org/patch/4682/Signed-off-by: NJohn Crispin <blogic@openwrt.org>

Display the MIPS ISA version release in the /proc/cpuinfo file.

[ralf@linux-mips.org: Add support for MIPS I ... IV legacy architecture
revisions.  Also differenciate between MIPS32 and MIPS64 versions instead
of lumping them together as just r1 and r2.

Note to application programmers: this indicates the CPU's ISA level
It does not imply the current execution environment does support it.  For
example an O32 application seeing "mips64r2" would still be restricted by
by the execution environment to 32-bit - but the kernel could run mips64r2
code.  The same for a 32-bit kernel running on a 64-bit processor.  This
field doesn't include ASEs or optional architecture modules nor other
detailed flags such as the availability of an FPU.]
Signed-off-by: NSteven J. Hill <sjhill@mips.com>
Cc: linux-mips@linux-mips.org
Patchwork: http://patchwork.linux-mips.org/patch/4714/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Having received another series of whitespace patches I decided to do this
once and for all rather than dealing with this kind of patches trickling
in forever.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

[ralf@linux-mips.org: This patch really only detects the ASE and passes its
existence on to userland via /proc/cpuinfo.  The DSP ASE Rev 2. adds new
resources but no resources that would need management by the kernel.]
Signed-off-by: NSteven J. Hill <sjhill@mips.com>
Cc: linux-mips@linux-mips.org
Patchwork: https://patchwork.linux-mips.org/patch/4165/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The PCI (Program Counter Interrupt) bit in the "cause" register
is mandatory for MIPS32R2 cores, but has also been added to some R1
cores (BMIPS5000). This change adds a cpu feature bit to make it
easier to check for and use this feature.
Signed-off-by: NAl Cooper <alcooperx@gmail.com>
Cc: linux-mips@linux-mips.org
Cc: linux-kernel@vger.kernel.org
Patchwork: https://patchwork.linux-mips.org/patch/4106/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Remove usage of the 'kernel_uses_smartmips_rixi' macro from all files
and use new 'cpu_has_rixi' instead.
Signed-off-by: NSteven J. Hill <sjhill@mips.com>
Acked-by: NDavid Daney <david.daney@cavium.com>

Originally both Read Inhibit (RI) and Execute Inhibit (XI) were
supported by the TLB only for a SmartMIPS core. The MIPSr3(TM)
Architecture now defines an optional feature to implement these
TLB bits separately. Support for one or both features can be
checked by looking at the Config3.RXI bit.
Signed-off-by: NSteven J. Hill <sjhill@mips.com>
Acked-by: NDavid Daney <david.daney@cavium.com>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The SmartMIPS ASE specifies how Read Inhibit (RI) and eXecute Inhibit
(XI) bits in the page tables work.  The upper two bits of EntryLo{0,1}
are RI and XI when the feature is enabled in the PageGrain register.
SmartMIPS only covers 32-bit systems.  Cavium Octeon+ extends this to
64-bit systems by continuing to place the RI and XI bits in the top of
EntryLo even when EntryLo is 64-bits wide.

Because we need to carry the RI and XI bits in the PTE, the layout of
the PTE is changed.  There is a two instruction overhead in the TLB
refill hot path to get the EntryLo bits into the proper position.
Also the TLB load exception has to probe the TLB to check if RI or XI
caused the exception.

Also of note is that the layout of the PTE bits is done at compile and
runtime rather than statically.  In the 32-bit case this allows for
the same number of PFN bits as before the patch as the _PAGE_HUGE is
not supported in 32-bit kernels (we have _PAGE_NO_EXEC and
_PAGE_NO_READ instead of _PAGE_READ and _PAGE_HUGE).

The patch is tested on Cavium Octeon+, but should also work on 32-bit
systems with the Smart-MIPS ASE.
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
To: linux-mips@linux-mips.org
Patchwork: http://patchwork.linux-mips.org/patch/952/
Patchwork: http://patchwork.linux-mips.org/patch/956/
Patchwork: http://patchwork.linux-mips.org/patch/962/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Linux kernel 2.6.32 and later allocate address space from the top of the
kernel virtual memory address space.

This patch implements virtual memory size detection for 64 bit MIPS CPUs
to avoid resulting crashes.
Signed-off-by: NGuenter Roeck <guenter.roeck@ericsson.com>
Cc: linux-mips@linux-mips.org
Patchwork: http://patchwork.linux-mips.org/patch/935/Reviewed-by: NDavid Daney <ddaney@caviumnetworks.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

On some CPUs, it is more efficient to disable and enable interrupts in the
kernel rather than use ll/sc for atomic operations.  But if we were to set
cpu_has_llsc to false, we would break the userspace futex interface (in
asm/futex.h).

We separate the two concepts, with a new predicate kernel_uses_llsc, that
lets us disable the kernel's use of ll/sc while still allowing the futex
code to use it.

Also there were a couple of cases in bitops.h where we were using ll/sc
unconditionally even if cpu_has_llsc were false.
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>