Calling ISA 3.0 instructions copy, copy_first, paste and paste_last
generates an alignment fault when copying or pasting unaligned
data (128 byte). We catch this and send SIGBUS to the userspace
process that caused it.

We do not emulate these because paste may contain additional metadata
when pasting to a co-processor and paste_last is the synchronisation
point for preceding copy/paste sequences.

Thanks to Michael Neuling <mikey@neuling.org> for his help.
Signed-off-by: NChris Smart <chris@distroguy.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

Sparse complains that it doesn't know what REG_BYTE is:

  arch/powerpc/kernel/align.c:313:29: error: undefined identifier 'REG_BYTE'

REG_BYTE is defined differently based on whether we're compiling for
LE, BE32 or BE64. Sparse apparently doesn't provide __BIG_ENDIAN__ or
__LITTLE_ENDIAN__, which means we get no definition.

Rather than check for __BIG_ENDIAN__ and then separately for
__LITTLE_ENDIAN__, just switch the #ifdef to check for __BIG_ENDIAN__
and then #else we define the little endian version. Technically that's
dicey because PDP_ENDIAN is also a possibility, but we already do it in
a lot of places so one more hardly matters.
Signed-off-by: NDaniel Axtens <dja@axtens.net>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

The enable_kernel_*() functions leave the relevant MSR bits enabled
until we exit the kernel sometime later. Create disable versions
that wrap the kernel use of FP, Altivec VSX or SPE.

While we don't want to disable it normally for performance reasons
(MSR writes are slow), it will be used for a debug boot option that
does this and catches bad uses in other areas of the kernel.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

Looks like I introduced this when adding LE support.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

Use make_dsisr instead of open coding it. This also have
the added benefit of handling alignment interrupt on additional
instructions.
Signed-off-by: NAneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: NAlexander Graf <agraf@suse.de>

Recent CPUs support quad word load and store instructions. Add
support to the alignment handler for them.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

This patch enables alignment handling for the load/store floating point
pair instructions (lfdp, lfdpx, stfdp, stfdpx).  The handler routine
is properly coded and only needs to be enabled.
Signed-off-by: NTom Musta <tmusta@gmail.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

The alignment handler is incorrect for unaligned string instructions
in little endian mode.  These instructions access data as arrays of
bytes and thus are endian neutral.  However, the routine also handles
the load/store multiple instructions, which are NOT endian neutral.

This patch toggles the byte swapping flag for the string instructions
in little endian builds.  This effectively disables the byte swapping
logic.
Signed-off-by: NTom Musta <tmusta@gmail.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

This creates new 'thread_fp_state' and 'thread_vr_state' structures
to store FP/VSX state (including FPSCR) and Altivec/VSX state
(including VSCR), and uses them in the thread_struct.  In the
thread_fp_state, the FPRs and VSRs are represented as u64 rather
than double, since we rarely perform floating-point computations
on the values, and this will enable the structures to be used
in KVM code as well.  Similarly FPSCR is now a u64 rather than
a structure of two 32-bit values.

This takes the offsets out of the macros such as SAVE_32FPRS,
REST_32FPRS, etc.  This enables the same macros to be used for normal
and transactional state, enabling us to delete the transactional
versions of the macros.   This also removes the unused do_load_up_fpu
and do_load_up_altivec, which were in fact buggy since they didn't
create large enough stack frames to account for the fact that
load_up_fpu and load_up_altivec are not designed to be called from C
and assume that their caller's stack frame is an interrupt frame.
Signed-off-by: NPaul Mackerras <paulus@samba.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Things are complicated by the fact that VSX elements are big
endian ordered even in little endian mode. 8 byte loads and
stores also write to the top 8 bytes of the register.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Handle most unaligned load and store faults in little
endian mode. Strings, multiples and VSX are not supported.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

The TS_FPR macro selects the FPR component of a VSX register (the
high doubleword). emulate_vsx is using this macro to get the
address of the associated VSX register. This happens to work on big
endian, but fails on little endian.

Replace it with an explicit array access.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

The alignment handler assumes big endian ordering when selecting
the low word of a 64bit floating point value. Use the existing
union which works in both little and big endian.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Use swab64/32/16 instead of open coding it.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

The VSX alignment handler needs to write out the existing VSX
state to memory before operating on it (flush_vsx_to_thread()).
If we take a VSX alignment exception in the kernel bad things
will happen. It looks like we could write the kernel state out
to the user process, or we could handle the kernel exception
using data from the user process (depending if MSR_VSX is set
or not).

Worse still, if the code to read or write the VSX state causes an
alignment exception, we will recurse forever. I ended up with
hundreds of megabytes of kernel stack to look through as a result.

Floating point and SPE code have similar issues but already include
a user check. Add the same check to emulate_vsx().

With this patch any unaligned VSX loads and stores in the kernel
will show up as a clear oops rather than silent corruption of
kernel or userspace VSX state, or worse, corruption of a potentially
unlimited amount of kernel memory.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Normally when we haven't implemented an alignment handler for
a load or store instruction the process will be terminated.

The alignment handler uses the DSISR (or a pseudo one) to locate
the right handler. Unfortunately ldbrx and stdbrx overlap lfs and
stfs so we incorrectly think ldbrx is an lfs and stdbrx is an
stfs.

This bug is particularly nasty - instead of terminating the
process we apply an incorrect fixup and continue on.

With more and more overlapping instructions we should stop
creating a pseudo DSISR and index using the instruction directly,
but for now add a special case to catch ldbrx/stdbrx.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Disintegrate asm/system.h for PowerPC.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
cc: linuxppc-dev@lists.ozlabs.org

Neither lfs nor stfs touch the fpscr, so remove the restore/save of it
around them.
Signed-off-by: NAndreas Schwab <schwab@linux-m68k.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

This patch fixes the handling of VSX alignment faults in little-endian
mode (the current code assumes the processor is in big-endian mode).

The patch also makes the handlers clear the top 8 bytes of the register
when handling an 8 byte VSX load.

This is based on 2.6.32.
Signed-off-by: NNeil Campbell <neilc@linux.vnet.ibm.com>
Cc: <stable@kernel.org>
Acked-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

perf_event wants a separate event for alignment and emulation faults,
so create another emulation event.  This will make it easy to hook in
perf_event at one spot.

We pass in regs which will be required for these events.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

If CONFIG_PPC_EMULATED_STATS is enabled, make available counters for the
various classes of emulated instructions under
/sys/kernel/debug/powerpc/emulated_instructions/ (assumed debugfs is mounted on
/sys/kernel/debug).  Optionally (controlled by
/sys/kernel/debug/powerpc/emulated_instructions/do_warn), rate-limited warnings
can be printed to the console when instructions are emulated.
Signed-off-by: NGeert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

When we introduced VSX, we changed the way FPRs are stored in the
thread_struct.  Unfortunately we missed the load/store float double
alignment handler code when updating how we access FPRs in the
thread_struct.

Below fixes this and merges the little/big endian case.
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

When we introduced VSX, we changed the way FPRs are stored in the
thread_struct.  Unfortunately we missed the load/store float double
alignment handler code when updating how we access FPRs in the
thread_struct.

Below fixes this and merges the little/big endian case.
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

lfiwzx is a new floating point load instruction in 2.06 that needs an
alignment handler for Linux.

Turns out to be the worlds easiest handler to add.
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Fix the VSX alignment handler for VSX registers > 32.  32-63 are stored
in the VMX part of the thread_struct not the FPR part.
Signed-off-by: NMichael Neuling <mikey@neuling.org>
CC: stable@kernel.org (2.6.27 & .28 please)
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

This fixes an uninitialised variable in the VSX alignment code.  It can
cause warnings from GCC (noticed with gcc-4.1.1).  Gcc is actually
correct in this instance, and this bug could cause the alignment
interrupt handler to send a SIGSEGV to the process on a legitimate
access.
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

VSX loads and stores will take an alignment exception when the address
is not on a 4 byte boundary.

This add support for these alignment exceptions and will emulate the
requested load or store.
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

regs is not used in emulate_fp_pair so remove it.
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

We are going to change where the floating point registers are stored
in the thread_struct, so in preparation add some macros to access the
floating point registers.  Update all code to use these new macros.
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This adds code to handle alignment traps generated by the following
SPE (signal processing engine) load/store instructions, by emulating
the instruction in the kernel (as is done for other instructions that
generate alignment traps):

evldd[x]         Vector Load Double Word into Double Word [Indexed]
evldw[x]         Vector Load Double into Two Words [Indexed]
evldh[x]         Vector Load Double into Four Half Words [Indexed]
evlhhesplat[x]   Vector Load Half Word into Half Words Even and Splat [Indexed]
evlhhousplat[x]  Vector Load Half Word into Half Word Odd Unsigned and Splat [Indexed]
evlhhossplat[x]  Vector Load Half Word into Half Word Odd Signed and Splat [Indexed]
evlwhe[x]        Vector Load Word into Two Half Words Even [Indexed]
evlwhou[x]       Vector Load Word into Two Half Words Odd Unsigned (zero-extended) [Indexed]
evlwhos[x]       Vector Load Word into Two Half Words Odd Signed (with sign extension) [Indexed]
evlwwsplat[x]    Vector Load Word into Word and Splat [Indexed]
evlwhsplat[x]    Vector Load Word into Two Half Words and Splat [Indexed]
evstdd[x]        Vector Store Double of Double [Indexed]
evstdw[x]        Vector Store Double of Two Words [Indexed]
evstdh[x]        Vector Store Double of Four Half Words [Indexed]
evstwhe[x]       Vector Store Word of Two Half Words from Even [Indexed]
evstwho[x]       Vector Store Word of Two Half Words from Odd [Indexed]
evstwwe[x]       Vector Store Word of Word from Even [Indexed]
evstwwo[x]       Vector Store Word of Word from Odd [Indexed]
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

This adds code to handle alignment traps generated by the following
new floating-point load/store instructions, by emulating the
instruction in the kernel (as is done for other instructions that
generate alignment traps):

lfiwax	load floating-point as integer word algebraic indexed
stfiwx	store floating-point as integer word indexed
lfdp	load floating-point double pair
lfdpx	load floating-point double pair indexed
stfdp	store floating-point double pair
stfdpx	store floating-point double pair indexed

All these except stfiwx are new in POWER6.

lfdp/lfdpx/stfdp/stfdpx load and store 16 bytes of memory into an
even/odd FP register pair.  In little-endian mode each 8-byte value is
byte-reversed separately (i.e. not as a 16-byte unit).  lfiwax/stfiwx
load or store the lower 4 bytes of a floating-point register from/to
memory; lfiwax sets the upper 4 bytes of the FP register to the sign
extension of the value loaded.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Make the alignment exception handler use the new _inatomic variants
of __get/put_user. This fixes erroneous warnings in the very rare
cases where we manage to have copy_tofrom_user_inatomic() trigger
an alignment exception.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

 arch/powerpc/kernel/align.c |   56 ++++++++++++++++++++++++--------------------
 1 file changed, 31 insertions(+), 25 deletions(-)
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This adds the PowerPC part of the code to allow processes to change
their endian mode via prctl.

This also extends the alignment exception handler to be able to fix up
alignment exceptions that occur in little-endian mode, both for
"PowerPC" little-endian and true little-endian.

We always enter signal handlers in big-endian mode -- the support for
little-endian mode does not amount to the creation of a little-endian
user/kernel ABI.  If the signal handler returns, the endian mode is
restored to what it was when the signal was delivered.

We have two new kernel CPU feature bits, one for PPC little-endian and
one for true little-endian.  Most of the classic 32-bit processors
support PPC little-endian, and this is reflected in the CPU feature
table.  There are two corresponding feature bits reported to userland
in the AT_HWCAP aux vector entry.

This is based on an earlier patch by Anton Blanchard.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch merges align.c, the result isn't quite what was in ppc64 nor
what was in ppc32 :) It should implement all the functionalities of both
though. Kumar, since you played with that in the past, I suppose you
have some test cases for verifying that it works properly before I dig
out the 601 machine ? :)

Since it's likely that I won't be able to test all scenario, code
inspection is much welcome.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The recent merge of fpu.S broken the handling of fpscr for
ARCH=powerpc and CONFIG_PPC64=y.  FP registers could be corrupted,
leading to strange random application crashes.

The confusion arises, because the thread_struct has (and requires) a
64-bit area to save the fpscr, because we use load/store double
instructions to get it in to/out of the FPU.  However, only the low
32-bits are actually used, so we want to treat it as a 32-bit quantity
when manipulating its bits to avoid extra load/stores on 32-bit.  This
patch replaces the current definition with a structure of two 32-bit
quantities (pad and val), to clarify things as much as is possible.
The 'val' field is used when manipulating bits, the structure itself
is used when obtaining the address for loading/unloading the value
from the FPU.

While we're at it, consolidate the 4 (!) almost identical versions of
cvt_fd() and cvt_df() (arch/ppc/kernel/misc.S,
arch/ppc64/kernel/misc.S, arch/powerpc/kernel/misc_32.S,
arch/powerpc/kernel/misc_64.S) into a single version in fpu.S.  The
new version takes a pointer to thread_struct and applies the correct
offset itself, rather than a pointer to the fpscr field itself, again
to avoid confusion as to which is the correct field to use.

Finally, this patch makes ARCH=ppc64 also use the consolidated fpu.S
code, which it previously did not.

Built for G5 (ARCH=ppc64 and ARCH=powerpc), 32-bit powermac (ARCH=ppc
and ARCH=powerpc) and Walnut (ARCH=ppc, CONFIG_MATH_EMULATION=y).
Booted on G5 (ARCH=powerpc) and things which previously fell over no
longer do.
Signed-off-by: NDavid Gibson <dwg@au1.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!