The SmartMIPS ASE specifies how Read Inhibit (RI) and eXecute Inhibit
(XI) bits in the page tables work.  The upper two bits of EntryLo{0,1}
are RI and XI when the feature is enabled in the PageGrain register.
SmartMIPS only covers 32-bit systems.  Cavium Octeon+ extends this to
64-bit systems by continuing to place the RI and XI bits in the top of
EntryLo even when EntryLo is 64-bits wide.

Because we need to carry the RI and XI bits in the PTE, the layout of
the PTE is changed.  There is a two instruction overhead in the TLB
refill hot path to get the EntryLo bits into the proper position.
Also the TLB load exception has to probe the TLB to check if RI or XI
caused the exception.

Also of note is that the layout of the PTE bits is done at compile and
runtime rather than statically.  In the 32-bit case this allows for
the same number of PFN bits as before the patch as the _PAGE_HUGE is
not supported in 32-bit kernels (we have _PAGE_NO_EXEC and
_PAGE_NO_READ instead of _PAGE_READ and _PAGE_HUGE).

The patch is tested on Cavium Octeon+, but should also work on 32-bit
systems with the Smart-MIPS ASE.
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
To: linux-mips@linux-mips.org
Patchwork: http://patchwork.linux-mips.org/patch/952/
Patchwork: http://patchwork.linux-mips.org/patch/956/
Patchwork: http://patchwork.linux-mips.org/patch/962/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

64-bit CPUs have 64-bit c0_entrylo{0,1} registers.  We should use the
64-bit dmtc0 instruction to set them.  This becomes important if we
want to set the RI and XI bits present in some processors.
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
To: linux-mips@linux-mips.org
Patchwork: http://patchwork.linux-mips.org/patch/954/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NFlorian Fainelli <florian@openwrt.org>
To: linux-mips@linux-mips.org
To: David Daney <ddaney@caviumnetworks.com>
Patchwork: http://patchwork.linux-mips.org/patch/887/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

For 64-bit kernels with 64KB pages and two level page tables, there are
42 bits worth of virtual address space This is larger than the 40 bits of
virtual address space obtained with the default 4KB Page size and three
levels, so there are no draw backs for using two level tables with this
configuration.
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Cc: linux-mips@linux-mips.org
Patchwork: http://patchwork.linux-mips.org/patch/761/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

commit c8af165342e83a4eb078c9607d29a7c399d30a53 (lmo) rsp.
e0cc87f5 (kernel.org) left
label_module_alloc unused.  Remove it now.
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Cc: linux-mips@linux-mips.org
Patchwork: http://patchwork.linux-mips.org/patch/752/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Processors that support the mips64r2 ISA can in four instructions
convert a shifted PGD pointer stored in the upper bits of c0_context
into a usable pointer.  By doing this we save a memory load and
associated potential cache miss in the TLB exception handlers.

Since the upper bits of c0_context were holding the CPU number, we
move this to the upper bits of c0_xcontext which doesn't have enough
bits to hold the PGD pointer, but has plenty for the CPU number.
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Todo: Nothing ever detects CPU_BCM6338 but the code tests for it anyway.
Signed-off-by: NMaxime Bizon <mbizon@freebox.fr>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

By combining swapper_pg_dir and module_pg_dir, several if conditions
can be eliminated from the tlb exception handler. The reason they
can be combined is that, the effective virtual address of vmalloc
returned is at the bottom, and of module_alloc returned is at the
top. It also fixes the bug in vmalloc(), which happens when its
return address is not covered by the first pgd.
Signed-off-by: NWu Fei <at.wufei@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Some of the were relying into smp.h being dragged in by another header
which of course is fragile.  <asm/cpu-info.h> uses smp_processor_id()
only in macros and including smp.h there leads to an include loop, so
don't change cpu-info.h.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The TLB handlers need to check for huge pages and give them special
handling.  Huge pages consist of two contiguous sub-pages of physical
memory.

* Loading entrylo0 and entrylo1 need to be handled specially.

* The page mask must be set for huge pages and then restored after
  writing the TLB entries.

* The PTE for huge pages resides in the PMD, we halt traversal of the
  tables there.
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The l parameter to iPTE_LW() is unused. Remove it and from some of its
callers as well.
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

CPU_CAVIUM_OCTEON is mips_r2 which is handled before the switch.  This
label in the switch statement is dead code, so we remove it.
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Reviewed by: David VomLehn <dvomlehn@cisco.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Some CPUs do not need ehb instructions after writing CP0 registers.
By allowing ehb generation to be overridden in
cpu-feature-overrides.h, we can save a few instructions in the TLB
handler hot paths.
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Try to fold the 64-bit TLB refill handler opportunistically at the
beginning of the vmalloc path so as to avoid splitting execution flow in
half and wasting cycles for a branch required at that point then.  Resort
to doing the split if either of the newly created parts would not fit into
its designated slot.
Original-patch-by: NMaciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The logic used to split the r4000 refill handler is liberally
sprinkled with magic numbers.  We attempt to explain what they are and
normalize them against a new symbolic value (MIPS64_REFILL_INSNS).

CC: David VomLehn <dvomlehn@cisco.com>
Reviewed-by: NPaul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The Alchemy manuals state:

"All pipeline hazards and dependencies are enforced by hardware interlocks
 so that any sequence of instructions is guaranteed to execute correctly.
 Therefore, it is not necessary to pad legacy MIPS hazards (such as
 load delay slots and coprocessor accesses) with NOPs."

Run-tested on Au12x0, without any ill effects.
Signed-off-by: NManuel Lauss <mano@roarinelk.homelinux.net>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

This patch removes the various CPU_AU1??? model constants in favor of
a single CPU_ALCHEMY one.

All currently existing Alchemy models are identical in terms of cpu
core and cache size/organization.  The parts of the mips kernel which
need to know the exact CPU revision extract it from the c0_prid register
already; and finally nothing else in-tree depends on those any more.

Should a new variant with slightly different "company options" and/or
"processor revision" bits in c0_prid appear, it will be supported
immediately (minus an exact model string in cpuinfo).
Signed-off-by: NManuel Lauss <mano@roarinelk.homelinux.net>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Current VR5500 processor support lacks of some functions which are
expected to be configured/synthesized on arch initialization.

Here're some VR5500A spec notes:

* All execution hazards are handled in hardware.

* Once VR5500A stops the operation of the pipeline by WAIT instruction,
  it could return from the standby mode only when either a reset, NMI
  request, or all enabled interrupts is/are detected.  In other words,
  if interrupts are disabled by Status.IE=0, it keeps in standby mode
  even when interrupts are internally asserted.

  Notes on WAIT: The operation of the processor is undefined if WAIT
  insn is in the branch delay slot.  The operation is also undefined
  if WAIT insn is executed when Status.EXL and Status.ERL are set to 1.

* VR5500A core only implements the Load prefetch.

With these changes, it boots fine.
Signed-off-by: NShinya Kuribayashi <shinya.kuribayashi@necel.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Expand the case statement for build_tlb_write_entry so that it does
the right thing on Cavium CPU variants.
Signed-off-by: NTomaso Paoletti <tpaoletti@caviumnetworks.com>
Signed-off-by: NPaul Gortmaker <Paul.Gortmaker@windriver.com>
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

trap_init issues flush_icache_range(), which uses ipi functions to
get icache flushing done on all cpus. But this is done before interrupts
are enabled and caused WARN_ON messages. This changeset introduces
a new local_flush_icache_range() and uses it before interrupts (and
additional CPUs) are enabled to avoid this problem.
Signed-off-by: NThomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Treat R4700 like R4600 in build_tlb_probe_entry. Without this fix kernel
will lock up.
Signed-off-by: NThomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Early 4KEc were MIPS32r1 and therefore need some love to get a TLB
refill handler.
Signed-off-by: NThomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

This patch moves the micro-assembler in a separate implementation, as
it is useful for further run-time optimizations. The only change in
behaviour is cutting down printk noise at kernel startup time.

Checkpatch complains about macro parameters which aren't protected by
parentheses. I believe this is a flaw in checkpatch, the paste operator
used in those macros won't work with parenthesised parameters.
Signed-off-by: NThiemo Seufer <ths@networkno.de>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

This patch adds IDs for new Au1200 variants: Au1210 and Au1250.
They are essentially identical to the Au1200 except for the Au1210
which has a different SoC-ID in the PRId register [bits 31:24].
The Au1250 is a "Au1200 V0.2".
Signed-off-by: NManuel Lauss <mano@roarinelk.homelinux.net>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

 This complements the generic R4000/R4400 errata workaround code and adds 
bits for the daddiu problem.  In most places it just modifies handwritten 
assembly code so that the assembler is allowed to use a temporary register 
as daddiu may now be treated as a macro that expands to a sequence of li 
and daddu.  It is the AT register or, where AT is unavailable or used 
explicitly for another purpose, an explicitly-named register is selected, 
using the .set at=<reg> feature added recently to gas.  This feature is 
only used if CONFIG_CPU_DADDI_WORKAROUNDS has been set, so if the 
workaround remains disabled, the required version of binutils stays 
unchanged.

 Similarly, daddiu instructions put in branch delay slots in noreorder 
fragments are now taken out of them and the assembler is allowed to 
reorder them itself as possible (which it does making the whole idea of 
scheduling them into delay slots manually questionable).

 Also in the very few places where such a simple conversion was not 
possible, a handcoded longer sequence is implemented.

 Other than that there are changes to code responsible for building the 
TLB fault and page clear/copy handlers to avoid daddiu as appropriate.  
These are only effective if the erratum is verified to be present at the 
run time.

 Finally there is a trivial update to __delay(), because it uses daddiu in 
a branch delay slot.
Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

This reverts commit aaf76a32.

As requested by ranck Bui-Huu <fbuihuu@gmail.com>.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

This saves a few k on systems which only ever ship with a single CPU type.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Note that the BCM4710 does not support the wait instruction, this
is not a mistake in the code.
    
It originally comes from the OpenWrt patches.
    
Cc: Michael Buesch <mb@bu3sch.de>
Cc: Felix Fietkau <nbd@openwrt.org>
Cc: Florian Schirmer <jolt@tuxbox.org>
Signed-off-by: NAurelien Jarno <aurelien@aurel32.net>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NMaciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-Off-By: NThiemo Seufer <ths@networkno.de>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-Off-By: NThiemo Seufer <ths@networkno.de>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>