L1 base address is now aligned so we can insert L1 index into r11 directly and
then preserve r10
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

Kernel MMU handling code handles validity of entries via _PMD_PRESENT which
corresponds to V bit in MD_TWC and MI_TWC. When the V bit is not set, MPC8xx
triggers TLBError exception. So we don't have to check that and branch ourself
to TLBError. We can set TLB entries with non present entries, remove all those
tests and let the 8xx handle it. This reduce the number of cycle when the
entries are valid which is the case most of the time, and doesn't significantly
increase the time for handling invalid entries.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

Since commit 33fb845a ("powerpc/8xx: Don't use MD_TWC for walk"), MD_EPN and
MD_TWC are not writen anymore in FixupDAR so saving r3 has become useless.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

On powerpc 8xx, in TLB entries, 0x400 bit is set to 1 for read-only pages
and is set to 0 for RW pages. So we should use _PAGE_RO instead of _PAGE_RW
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

8xx sometimes need to load a invalid/non-present TLBs in
it DTLB asm handler.

These must be invalidated separaly as linux mm doesn't.

Commit 5efab4a0 was invalidating them in
arch/powerpc/mm/fault.c.
This patch does the invalidation earlier in order to free the TLB as soon as
possible. This also has the advantage of removing some 8xx specific code from
fault.c
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

As we are not using anymore DAR to save registers, it is now available for
saving the r3 register used for CPU6 ERRATA handling. Therefore we can
remove the major hack which was to use memory location 0 to save r3.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

There is not need to restore r10, r11 and cr registers at this end of ITLBmiss
handler as they are saved again to the same place in ITLBError handler we are
jumping to.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

When a PMD entry is valid, _PMD_PRESENT is set. Therefore, forcing that bit
during TLB loading is useless.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

No need to re-set this bit at each TLB miss. Let's set it in the PTE.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

This patch hiddes that SPR address needed for CPU6 ERRATA handling in the macro.
Then we don't have to worry about this address directly in the code.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

This patch activates the handling of 16k pages on the MPC8xx.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

Value 0x00f0 is used to force bits in TLB level 2 entry. This value is linked
to the page size and will vary when we change the page size. Lets define a const
for it in order to have it at only one place.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

For PAGE size related operations, use PAGE size consts in order to be able to
use different page size in the futur.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

MD_TWC can only be used properly with 4k pages.
So lets calculate level 2 table index by ourselves.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

Use M_TW instead of M_TWB for storing Level 1 table address as M_TWB requires
4k aligned tables, which is only the case with 4k pages.
Consequently, we have to calculate the level 1 table index by ourselves.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

In DTLBError handler there is not need to restore r10, r11 and cr registers
after fixing DAR as they are saved again to the same place just after.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

DataAccess exception is never generated by MPC8xx so do the job directly where
it is used to avoid an unnecessary branching.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

Exception InstructionAccess does not exist on MPC8xx. No need to branch there from somewhere else.
Handling can be done directly in InstructionTLBError Exception.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

Branching takes two cycles on MPC8xx. Lets duplicate the two instructions
and avoid the branching.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

By XORing the upper part of the instruction code, we get a value that can
directly be verified with the second test and we can remove the first test.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

r10 and r3 are only used inside FixupDAR function. So lets save them inside
that function only.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

Since commit 2321f337, dirty handling is not
handled here anymore. So we fix the comment.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

Since commit 2321f337, r10 is not used anymore
after FixupDAR. There is therefore no need to set it up with the value of DAR.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

SCRATCH0 and SCRATCH1 are only used in Exceptions prologs where no other
exception can happen. There is therefore no need to preserve them accross
TLB handlers, we can use them there as in other exceptions. One of the
advantages is that they do not suffer CPU6 errata unlike M_TW register.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

Since commit 469d62be, SPRG2 is used as a
scratch register just like SPRG0 and SPRG1. So Declare it as such and fix
the comment which is not valid anymore since that commit.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

Activating CONFIG_PIN_TLB is supposed to pin the IMMR and the first
three 8Mbytes pages.  But the setting of MD_CTR to a pinnable entry was
missing before the pinning of the third 8Mb page.  As the index is
decremented module 28 (MD_RSV4D is set) after every DTLB update, the
third 8Mbytes page was not pinned.
Signed-off-by: NChristophe Leroy <christophe.leroy@c-s.fr>
Signed-off-by: NScott Wood <scottwood@freescale.com>

Instead of implementing an empty giveup_fpu() function for each
32bit processor type, replace them with an unique empty inline
function.
Signed-off-by: NKevin Hao <haokexin@gmail.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

We currently turn interrupts back to their previous state before
calling do_page_fault(). This can be annoying when debugging as
a bad fault will potentially have lost some processor state before
getting into the debugger.

We also end up calling some generic code with interrupts enabled
such as notify_page_fault() with interrupts enabled, which could
be unexpected.

This changes our code to behave more like other architectures,
and make the assembly entry code call into do_page_faults() with
interrupts disabled. They are conditionally re-enabled from
within do_page_fault() in the same spot x86 does it.

While there, add the might_sleep() test in the case of a successful
trylock of the mmap semaphore, again like x86.

Also fix a bug in the existing assembly where r12 (_MSR) could get
clobbered by C calls (the DTL accounting in the exception common
macro and DISABLE_INTS) in some cases.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
---

v2. Add the r12 clobber fix

u64 is used rather than phys_addr_t to keep things simple, as
this is called from assembly code.

Update callers to pass a 64-bit address in r3/r4.  Other unused
register assignments that were once parameters to machine_init
are dropped.

For FSL BookE, look up the physical address of the device tree from the
effective address passed in r3 by the loader.  This is required for
situations where memory does not start at zero (due to AMP or IOMMU-less
virtualization), and thus the IMA doesn't start at zero, and thus the
device tree effective address does not equal the physical address.
Signed-off-by: NScott Wood <scottwood@freescale.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Since STACK_FRAME_OVERHEAD is defined in asm/ptrace.h and that
is ASSEMBER safe, we can just include that instead of going via
asm-offsets.h.
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

This avoids storing these registers in memory.
CPU6 errata will still use the old way.
Remove some G2 leftover accesses from 2.4
Signed-off-by: NJoakim Tjernlund <Joakim.Tjernlund@transmode.se>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Only the swap function cares about the ACCESSED bit in
the pte. Do not waste cycles updateting ACCESSED when swap
is not compiled into the kernel.
Signed-off-by: NJoakim Tjernlund <Joakim.Tjernlund@transmode.se>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Only modules will cause ITLB Misses as we always pin
the first 8MB of kernel memory.
Signed-off-by: NJoakim Tjernlund <Joakim.Tjernlund@transmode.se>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

This removes a couple of insn's from the TLB Miss
handlers whithout changing functionality.
Signed-off-by: NJoakim Tjernlund <Joakim.Tjernlund@transmode.se>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

The newly added fixup for buggy dcbX insn's has
a bug that always trigger a kernel TLB walk so a user space
dcbX insn will cause a Kernel Machine Check if it hits DTLB error.
Signed-off-by: NJoakim Tjernlund <Joakim.Tjernlund@transmode.se>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Various kernel asm modifies SRR0/SRR1 just before executing
a rfi. If such code crosses a page boundary you risk a TLB miss
which will clobber SRR0/SRR1. Avoid this by always pinning
kernel instruction TLB space.
Signed-off-by: NJoakim Tjernlund <Joakim.Tjernlund@transmode.se>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Use symbolic constant for PRESENT and avoid branching.
Signed-off-by: NJoakim Tjernlund <Joakim.Tjernlund@transmode.se>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

There is no need to do set the DIRTY bit directly in DTLB Error.
Trap to do_page_fault() and let the generic MM code do the work.
Signed-off-by: NJoakim Tjernlund <Joakim.Tjernlund@transmode.se>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

8xx has not had WRITETHRU due to lack of bits in the pte.
After the recent rewrite of the 8xx TLB code, there are
two bits left. Use one of them to WRITETHRU.

Perhaps use the last SW bit to PAGE_SPECIAL or PAGE_FILE?
Signed-off-by: NJoakim Tjernlund <Joakim.Tjernlund@transmode.se>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

only DTLB Miss did set this bit, DTLB Error needs too otherwise
the setting is lost when the page becomes dirty.
Signed-off-by: NJoakim Tjernlund <Joakim.Tjernlund@transmode.se>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>