mainline inclusion
from mainline-v5.15-rc6
commit b2381acd
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit b2381acd x86/fpu: Mask out the invalid MXCSR bits properly.

--------------------------------

This is a fix for the fix (yeah, /facepalm).

The correct mask to use is not the negation of the MXCSR_MASK but the
actual mask which contains the supported bits in the MXCSR register.

Reported and debugged by Ville Syrjälä <ville.syrjala@linux.intel.com>

Fixes: d298b035 ("x86/fpu: Restore the masking out of reserved MXCSR bits")
Signed-off-by: NBorislav Petkov <bp@suse.de>
Tested-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Tested-by: NSer Olmy <ser.olmy@protonmail.com>
Cc: <stable@vger.kernel.org>
Link: https://lore.kernel.org/r/YWgYIYXLriayyezv@intel.comSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.15-rc5
commit d298b035
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit d298b035 x86/fpu: Restore the masking out of reserved MXCSR bits.

--------------------------------

Ser Olmy reported a boot failure:

  init[1] bad frame in sigreturn frame:(ptrval) ip:b7c9fbe6 sp:bf933310 orax:ffffffff \
	  in libc-2.33.so[b7bed000+156000]
  Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
  CPU: 0 PID: 1 Comm: init Tainted: G        W         5.14.9 #1
  Hardware name: Hewlett-Packard HP PC/HP Board, BIOS  JD.00.06 12/06/2001
  Call Trace:
   dump_stack_lvl
   dump_stack
   panic
   do_exit.cold
   do_group_exit
   get_signal
   arch_do_signal_or_restart
   ? force_sig_info_to_task
   ? force_sig
   exit_to_user_mode_prepare
   syscall_exit_to_user_mode
   do_int80_syscall_32
   entry_INT80_32

on an old 32-bit Intel CPU:

  vendor_id       : GenuineIntel
  cpu family      : 6
  model           : 6
  model name      : Celeron (Mendocino)
  stepping        : 5
  microcode       : 0x3

Ser bisected the problem to the commit in Fixes.

tglx suggested reverting the rejection of invalid MXCSR values which
this commit introduced and replacing it with what the old code did -
simply masking them out to zero.

Further debugging confirmed his suggestion:

  fpu->state.fxsave.mxcsr: 0xb7be13b4, mxcsr_feature_mask: 0xffbf
  WARNING: CPU: 0 PID: 1 at arch/x86/kernel/fpu/signal.c:384 __fpu_restore_sig+0x51f/0x540

so restore the original behavior only for 32-bit kernels where you have
ancient machines with buggy hardware. For 32-bit programs on 64-bit
kernels, user space which supplies wrong MXCSR values is considered
malicious so fail the sigframe restoration there.

Fixes: 6f9866a1 ("x86/fpu/signal: Let xrstor handle the features to init")
Reported-by: NSer Olmy <ser.olmy@protonmail.com>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Tested-by: NSer Olmy <ser.olmy@protonmail.com>
Cc: <stable@vger.kernel.org>
Link: https://lkml.kernel.org/r/YVtA67jImg3KlBTw@zn.tnicSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 93c2cdc9
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 93c2cdc9 x86/fpu/xstate: Clear xstate header in copy_xstate_to_uabi_buf() again.

--------------------------------

The change which made copy_xstate_to_uabi_buf() usable for
[x]fpregs_get() removed the zeroing of the header which means the
header, which is copied to user space later, contains except for the
xfeatures member, random stack content.

Add the memset() back to zero it before usage.

Fixes: eb6f5172 ("x86/fpu: Make copy_xstate_to_kernel() usable for [x]fpregs_get()")
Reported-by: Nkernel test robot <oliver.sang@intel.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/875yy3wb8h.ffs@nanos.tec.linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 6f9866a1
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 6f9866a1 x86/fpu/signal: Let xrstor handle the features to init.

--------------------------------

There is no reason to do an extra XRSTOR from init_fpstate for feature
bits which have been cleared by user space in the FX magic xfeatures
storage.

Just clear them in the task's XSTATE header and do a full restore which
will put these cleared features into init state.

There is no real difference in performance because the current code
already does a full restore when the xfeatures bits are preserved as the
signal frame setup has stored them, which is the full UABI feature set.

 [ bp: Use the negated mxcsr_feature_mask in the MXCSR check. ]
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121457.804115017@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit fcb3635f
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit fcb3635f x86/fpu/signal: Handle #PF in the direct restore path.

--------------------------------

If *RSTOR raises an exception, then the slow path is taken. That's wrong
because if the reason was not #PF then going through the slow path is waste
of time because that will end up with the same conclusion that the data is
invalid.

Now that the wrapper around *RSTOR return an negative error code, which is
the negated trap number, it's possible to differentiate.

If the *RSTOR raised #PF then handle it directly in the fast path and if it
was some other exception, e.g. #GP, then give up and do not try the fast
path.

This removes the legacy frame FRSTOR code from the slow path because FRSTOR
is not a ia32_fxstate frame and is therefore handled in the fast path.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121457.696022863@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 0a6c2e9e
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 0a6c2e9e x86/fpu/signal: Split out the direct restore code.

--------------------------------

Prepare for smarter failure handling of the direct restore.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121457.493455414@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit cdcec1b7
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit cdcec1b7 x86/fpu/signal: Sanitize copy_user_to_fpregs_zeroing().

--------------------------------

Now that user_xfeatures is correctly set when xsave is enabled, remove
the duplicated initialization of components.

Rename the function while at it.

Intel-SIG: commit cdcec1b7 x86/fpu/signal: Sanitize
copy_user_to_fpregs_zeroing().
Intel Advanced Matrix Extensions(AMX) is one of the key features on Sapphire
Rapids(SPR) and it depends on xsave, this patch is part of Thomas Gleixner's
x86 FPU code cleanup that upstream AMX implementation is building on, and is
also key to all code for being developed in upstream that depends on xsave.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121457.377341297@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 1258a8c8
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 1258a8c8 x86/fpu/signal: Sanitize the xstate check on sigframe.

--------------------------------

Utilize the check for the extended state magic in the FX software reserved
bytes and set the parameters for restoring fx_only in the relevant members
of fw_sw_user.

This allows further cleanups on top because the data is consistent.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121457.277738268@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 9ba589f9
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 9ba589f9 x86/fpu/signal: Remove the legacy alignment check.

--------------------------------

Checking for the XSTATE buffer being 64-byte aligned, and if not,
deciding just to restore the FXSR state is daft.

If user space provides an unaligned math frame and has the extended state
magic set in the FX software reserved bytes, then it really can keep the
pieces.

If the frame is unaligned and the FX software magic is not set, then
fx_only is already set and the restore will use fxrstor.

Remove it.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121457.184149902@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 99a59019
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 99a59019 x86/fpu/signal: Move initial checks into fpu__restore_sig().

--------------------------------

__fpu__restore_sig() is convoluted and some of the basic checks can
trivially be done in the calling function as well as the final error
handling of clearing user state.

 [ bp: Fixup typos. ]
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121457.086336154@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit bf68a7d9
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit bf68a7d9 x86/fpu: Mark init_fpstate __ro_after_init.

--------------------------------

Nothing has to write into that state after init.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121456.992342060@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 72a6c08c
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 72a6c08c x86/pkru: Remove xstate fiddling from write_pkru().

--------------------------------

The PKRU value of a task is stored in task->thread.pkru when the task is
scheduled out. PKRU is restored on schedule in from there. So keeping the
XSAVE buffer up to date is a pointless exercise.

Remove the xstate fiddling and cleanup all related functions.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121456.897372712@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 0e8c54f6
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 0e8c54f6 x86/fpu: Don't store PKRU in xstate in fpu_reset_fpstate().

--------------------------------

PKRU for a task is stored in task->thread.pkru when the task is scheduled
out. For 'current' the authoritative source of PKRU is the hardware.

fpu_reset_fpstate() has two callers:

  1) fpu__clear_user_states() for !FPU systems. For those PKRU is irrelevant

  2) fpu_flush_thread() which is invoked from flush_thread(). flush_thread()
     resets the hardware to the kernel restrictive default value.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121456.802850233@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 95443698
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 95443698 x86/fpu: Remove PKRU handling from switch_fpu_finish().

--------------------------------

PKRU is already updated and the xstate is not longer the proper source
of information.

 [ bp: Use cpu_feature_enabled() ]
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121456.708180184@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 30a304a1
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 30a304a1 x86/fpu: Mask PKRU from kernel XRSTOR[S] operations.

--------------------------------

As the PKRU state is managed separately restoring it from the xstate
buffer would be counterproductive as it might either restore a stale
value or reinit the PKRU state to 0.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121456.606745195@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit e84ba47e
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit e84ba47e x86/fpu: Hook up PKRU into ptrace().

--------------------------------

One nice thing about having PKRU be XSAVE-managed is that it gets naturally
exposed into the XSAVE-using ABIs.  Now that XSAVE will not be used to
manage PKRU, these ABIs need to be manually enabled to deal with PKRU.

ptrace() uses copy_uabi_xstate_to_kernel() to collect the tracee's
XSTATE. As PKRU is not in the task's XSTATE buffer, use task->thread.pkru
for filling in up the ptrace buffer.
Signed-off-by: NDave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121456.508770763@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 9782a712
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 9782a712 x86/fpu: Add PKRU storage outside of task XSAVE buffer.

--------------------------------

PKRU is currently partly XSAVE-managed and partly not. It has space
in the task XSAVE buffer and is context-switched by XSAVE/XRSTOR.
However, it is switched more eagerly than FPU because there may be a
need for PKRU to be up-to-date for things like copy_to/from_user() since
PKRU affects user-permission memory accesses, not just accesses from
userspace itself.

This leaves PKRU in a very odd position. XSAVE brings very little value
to the table for how Linux uses PKRU except for signal related XSTATE
handling.

Prepare to move PKRU away from being XSAVE-managed. Allocate space in
the thread_struct for it and save/restore it in the context-switch path
separately from the XSAVE-managed features. task->thread_struct.pkru
is only valid when the task is scheduled out. For the current task the
authoritative source is the hardware, i.e. it has to be retrieved via
rdpkru().

Leave the XSAVE code in place for now to ensure bisectability.
Signed-off-by: NDave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121456.399107624@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 2ebe81c6
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 2ebe81c6 x86/fpu: Dont restore PKRU in fpregs_restore_userspace().

--------------------------------

switch_to() and flush_thread() write the task's PKRU value eagerly so
the PKRU value of current is always valid in the hardware.

That means there is no point in restoring PKRU on exit to user or when
reactivating the task's FPU registers in the signal frame setup path.

This allows to remove all the xstate buffer updates with PKRU values once
the PKRU state is stored in thread struct while a task is scheduled out.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121456.303919033@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 65e95210
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 65e95210 x86/fpu: Rename xfeatures_mask_user() to xfeatures_mask_uabi().

--------------------------------

Rename it so it's clear that this is about user ABI features which can
differ from the feature set which the kernel saves and restores because the
kernel handles e.g. PKRU differently. But the user ABI (ptrace, signal
frame) expects it to be there.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121456.211585137@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 1d9bffab
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 1d9bffab x86/fpu: Move FXSAVE_LEAK quirk into __copy_kernel_to_fpregs().

--------------------------------

copy_kernel_to_fpregs() restores all xfeatures but it is also the place
where the AMD FXSAVE_LEAK bug is handled.

That prevents fpregs_restore_userregs() to limit the restored features,
which is required to untangle PKRU and XSTATE handling and also for the
upcoming supervisor state management.

Move the FXSAVE_LEAK quirk into __copy_kernel_to_fpregs() and deinline that
function which has become rather fat.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121456.114271278@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 727d0110
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 727d0110 x86/fpu: Rename __fpregs_load_activate() to fpregs_restore_userregs().

--------------------------------

Rename it so that it becomes entirely clear what this function is
about. It's purpose is to restore the FPU registers to the state which was
saved in the task's FPU memory state either at context switch or by an in
kernel FPU user.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121456.018867925@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 33344368
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 33344368 x86/fpu: Clean up the fpu__clear() variants.

--------------------------------

fpu__clear() currently resets both register state and kernel XSAVE buffer
state.  It has two modes: one for all state (supervisor and user) and
another for user state only.  fpu__clear_all() uses the "all state"
(user_only=0) mode, while a number of signal paths use the user_only=1
mode.

Make fpu__clear() work only for user state (user_only=1) and remove the
"all state" (user_only=0) code.  Rename it to match so it can be used by
the signal paths.

Replace the "all state" (user_only=0) fpu__clear() functionality.  Use the
TIF_NEED_FPU_LOAD functionality instead of making any actual hardware
registers changes in this path.

Instead of invoking fpu__initialize() just memcpy() init_fpstate into the
task's FPU state because that has already the correct format and in case of
PKRU also contains the default PKRU value. Move the actual PKRU write out
into flush_thread() where it belongs and where it will end up anyway when
PKRU and XSTATE have been untangled.

For bisectability a workaround is required which stores the PKRU value in
the xstate memory until PKRU is untangled from XSTATE for context
switching and return to user.

[ Dave Hansen: Polished changelog ]
[ tglx: Fixed the PKRU fallout ]
Signed-off-by: NAndy Lutomirski <luto@kernel.org>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121455.922729522@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit e7ecad17
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit e7ecad17 x86/fpu: Rename fpu__clear_all() to fpu_flush_thread().

--------------------------------

Make it clear what the function is about.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121455.827979263@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 37107113
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 37107113 x86/fpu: Use pkru_write_default() in copy_init_fpstate_to_fpregs().

--------------------------------

There is no point in using copy_init_pkru_to_fpregs() which in turn calls
write_pkru(). write_pkru() tries to fiddle with the task's xstate buffer
for nothing because the XRSTOR[S](init_fpstate) just cleared the xfeature
flag in the xstate header which makes get_xsave_addr() fail.

It's a useless exercise anyway because the reinitialization activates the
FPU so before the task's xstate buffer can be used again a XRSTOR[S] must
happen which in turn dumps the PKRU value.

Get rid of the now unused copy_init_pkru_to_fpregs().
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121455.732508792@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit fa8c84b7
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit fa8c84b7 x86/cpu: Write the default PKRU value when enabling PKE.

--------------------------------

In preparation of making the PKRU management more independent from XSTATES,
write the default PKRU value into the hardware right after enabling PKRU in
CR4. This ensures that switch_to() and copy_thread() have the correct
setting for init task and the per CPU idle threads right away.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121455.622983906@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit ff7ebff4
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit ff7ebff4 x86/pkru: Provide pkru_write_default().

--------------------------------

Provide a simple and trivial helper which just writes the PKRU default
value without trying to fiddle with the task's xsave buffer.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121455.513729794@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 739e2eec
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 739e2eec x86/pkru: Provide pkru_get_init_value().

--------------------------------

When CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS is disabled then the following
code fails to compile:

     if (cpu_feature_enabled(X86_FEATURE_OSPKE)) {
     	u32 pkru = READ_ONCE(init_pkru_value);
	..
     }

because init_pkru_value is defined as '0' which makes READ_ONCE() upset.

Provide an accessor macro to avoid #ifdeffery all over the place.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121455.404880646@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 8a1dc55a
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 8a1dc55a x86/cpu: Sanitize X86_FEATURE_OSPKE.

--------------------------------

X86_FEATURE_OSPKE is enabled first on the boot CPU and the feature flag is
set. Secondary CPUs have to enable CR4.PKE as well and set their per CPU
feature flag. That's ineffective because all call sites have checks for
boot_cpu_data.

Make it smarter and force the feature flag when PKU is enabled on the boot
cpu which allows then to use cpu_feature_enabled(X86_FEATURE_OSPKE) all
over the place. That either compiles the code out when PKEY support is
disabled in Kconfig or uses a static_cpu_has() for the feature check which
makes a significant difference in hotpaths, e.g. context switch.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121455.305113644@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit b2681e79
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit b2681e79 x86/fpu: Rename and sanitize fpu__save/copy().

--------------------------------

Both function names are a misnomer.

fpu__save() is actually about synchronizing the hardware register state
into the task's memory state so that either coredump or a math exception
handler can inspect the state at the time where the problem happens.

The function guarantees to preserve the register state, while "save" is a
common terminology for saving the current state so it can be modified and
restored later. This is clearly not the case here.

Rename it to fpu_sync_fpstate().

fpu__copy() is used to clone the current task's FPU state when duplicating
task_struct. While the register state is a copy the rest of the FPU state
is not.

Name it accordingly and remove the really pointless @src argument along
with the warning which comes along with it.

Nothing can ever copy the FPU state of a non-current task. It's clearly
just a consequence of arch_dup_task_struct(), but it makes no sense to
proliferate that further.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121455.196727450@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 784a4661
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 784a4661 x86/pkeys: Move read_pkru() and write_pkru().

--------------------------------

write_pkru() was originally used just to write to the PKRU register.  It
was mercifully short and sweet and was not out of place in pgtable.h with
some other pkey-related code.

But, later work included a requirement to also modify the task XSAVE
buffer when updating the register.  This really is more related to the
XSAVE architecture than to paging.

Move the read/write_pkru() to asm/pkru.h.  pgtable.h won't miss them.
Signed-off-by: NDave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121455.102647114@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit a75c5289
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit a75c5289 x86/fpu/xstate: Sanitize handling of independent features.

--------------------------------

The copy functions for the independent features are horribly named and the
supervisor and independent part is just overengineered.

The point is that the supplied mask has either to be a subset of the
independent features or a subset of the task->fpu.xstate managed features.

Rewrite it so it checks for invalid overlaps of these areas in the caller
supplied feature mask. Rename it so it follows the new naming convention
for these operations. Mop up the function documentation.

This allows to use that function for other purposes as well.
Suggested-by: NPeter Zijlstra <peterz@infradead.org>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Tested-by: NKan Liang <kan.liang@linux.intel.com>
Link: https://lkml.kernel.org/r/20210623121455.004880675@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 01707b66
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 01707b66 x86/fpu: Rename "dynamic" XSTATEs to "independent".

--------------------------------

The salient feature of "dynamic" XSTATEs is that they are not part of the
main task XSTATE buffer.  The fact that they are dynamically allocated is
irrelevant and will become quite confusing when user math XSTATEs start
being dynamically allocated.  Rename them to "independent" because they
are independent of the main XSTATE code.

This is just a search-and-replace with some whitespace updates to keep
things aligned.
Signed-off-by: NAndy Lutomirski <luto@kernel.org>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Link: https://lore.kernel.org/r/1eecb0e4f3e07828ebe5d737ec77dc3b708fad2d.1623388344.git.luto@kernel.org
Link: https://lkml.kernel.org/r/20210623121454.911450390@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit b76411b1
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit b76411b1 x86/fpu: Rename initstate copy functions.

--------------------------------

Again this not a copy. It's restoring register state from kernel memory.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121454.816581630@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 1c61fada
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 1c61fada x86/fpu: Rename copy_kernel_to_fpregs() to restore_fpregs_from_fpstate().

--------------------------------

This is not a copy functionality. It restores the register state from the
supplied kernel buffer.

No functional changes.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121454.716058365@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 08ded2cd
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 08ded2cd x86/fpu: Get rid of the FNSAVE optimization.

--------------------------------

The FNSAVE support requires conditionals in quite some call paths because
FNSAVE reinitializes the FPU hardware. If the save has to preserve the FPU
register state then the caller has to conditionally restore it from memory
when FNSAVE is in use.

This also requires a conditional in context switch because the restore
avoidance optimization cannot work with FNSAVE. As this only affects 20+
years old CPUs there is really no reason to keep this optimization
effective for FNSAVE. It's about time to not optimize for antiques anymore.

Just unconditionally FRSTOR the save content to the registers and clean up
the conditionals all over the place.
Suggested-by: NDave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121454.617369268@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit ebe7234b
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit ebe7234b x86/fpu: Rename copy_fpregs_to_fpstate() to save_fpregs_to_fpstate().

--------------------------------

A copy is guaranteed to leave the source intact, which is not the case when
FNSAVE is used as that reinitilizes the registers.

Save does not make such guarantees and it matches what this is about,
i.e. to save the state for a later restore.

Rename it to save_fpregs_to_fpstate().
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121454.508853062@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 522e9274
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 522e9274 x86/fpu: Deduplicate copy_uabi_from_user/kernel_to_xstate().

--------------------------------

copy_uabi_from_user_to_xstate() and copy_uabi_from_kernel_to_xstate() are
almost identical except for the copy function.

Unify them.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Acked-by: NAndy Lutomirski <luto@kernel.org>
Link: https://lkml.kernel.org/r/20210623121454.414215896@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 1cc34413
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 1cc34413 x86/fpu: Rename xstate copy functions which are related to UABI.

--------------------------------

Rename them to reflect that these functions deal with user space format
XSAVE buffers.

      copy_kernel_to_xstate() -> copy_uabi_from_kernel_to_xstate()
      copy_user_to_xstate()   -> copy_sigframe_from_user_to_xstate()

Again a clear statement that these functions deal with user space ABI.
Suggested-by: NAndy Lutomirski <luto@kernel.org>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121454.318485015@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 6fdc908c
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 6fdc908c x86/fpu: Rename fregs-related copy functions.

--------------------------------

The function names for fnsave/fnrstor operations are horribly named and
a permanent source of confusion.

Rename:
	copy_kernel_to_fregs() to frstor()
	copy_fregs_to_user()   to fnsave_to_user_sigframe()
	copy_user_to_fregs()   to frstor_from_user_sigframe()

so it's clear what these are doing. All these functions are really low
level wrappers around the equally named instructions, so mapping to the
documentation is just natural.

No functional change.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121454.223594101@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>

mainline inclusion
from mainline-v5.14-rc1
commit 872c65db
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I590ZC
CVE: NA

Intel-SIG: commit 872c65db x86/math-emu: Rename frstor().

--------------------------------

This is in the way of renaming the low level hardware accessors to match
the instruction name. Prepend it with FPU_ which is consistent vs. the
rest of the emulation code.

No functional change.

  [ bp: Correct the Reported-by: ]
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210623121454.111665161@linutronix.deSigned-off-by: NLin Wang <lin.x.wang@intel.com>
Signed-off-by: NAichun Shi <aichun.shi@intel.com>