1. 01 12月, 2019 1 次提交
    • M
      powerpc/book3s64: Fix link stack flush on context switch · 0a60d4bd
      Michael Ellerman 提交于
      commit 39e72bf96f5847ba87cc5bd7a3ce0fed813dc9ad upstream.
      
      In commit ee13cb24 ("powerpc/64s: Add support for software count
      cache flush"), I added support for software to flush the count
      cache (indirect branch cache) on context switch if firmware told us
      that was the required mitigation for Spectre v2.
      
      As part of that code we also added a software flush of the link
      stack (return address stack), which protects against Spectre-RSB
      between user processes.
      
      That is all correct for CPUs that activate that mitigation, which is
      currently Power9 Nimbus DD2.3.
      
      What I got wrong is that on older CPUs, where firmware has disabled
      the count cache, we also need to flush the link stack on context
      switch.
      
      To fix it we create a new feature bit which is not set by firmware,
      which tells us we need to flush the link stack. We set that when
      firmware tells us that either of the existing Spectre v2 mitigations
      are enabled.
      
      Then we adjust the patching code so that if we see that feature bit we
      enable the link stack flush. If we're also told to flush the count
      cache in software then we fall through and do that also.
      
      On the older CPUs we don't need to do do the software count cache
      flush, firmware has disabled it, so in that case we patch in an early
      return after the link stack flush.
      
      The naming of some of the functions is awkward after this patch,
      because they're called "count cache" but they also do link stack. But
      we'll fix that up in a later commit to ease backporting.
      
      This is the fix for CVE-2019-18660.
      Reported-by: NAnthony Steinhauser <asteinhauser@google.com>
      Fixes: ee13cb24 ("powerpc/64s: Add support for software count cache flush")
      Cc: stable@vger.kernel.org # v4.4+
      Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
      Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      0a60d4bd
  2. 06 4月, 2019 1 次提交
    • N
      powerpc/64s: Clear on-stack exception marker upon exception return · b52681e6
      Nicolai Stange 提交于
      [ Upstream commit eddd0b332304d554ad6243942f87c2fcea98c56b ]
      
      The ppc64 specific implementation of the reliable stacktracer,
      save_stack_trace_tsk_reliable(), bails out and reports an "unreliable
      trace" whenever it finds an exception frame on the stack. Stack frames
      are classified as exception frames if the STACK_FRAME_REGS_MARKER
      magic, as written by exception prologues, is found at a particular
      location.
      
      However, as observed by Joe Lawrence, it is possible in practice that
      non-exception stack frames can alias with prior exception frames and
      thus, that the reliable stacktracer can find a stale
      STACK_FRAME_REGS_MARKER on the stack. It in turn falsely reports an
      unreliable stacktrace and blocks any live patching transition to
      finish. Said condition lasts until the stack frame is
      overwritten/initialized by function call or other means.
      
      In principle, we could mitigate this by making the exception frame
      classification condition in save_stack_trace_tsk_reliable() stronger:
      in addition to testing for STACK_FRAME_REGS_MARKER, we could also take
      into account that for all exceptions executing on the kernel stack
        - their stack frames's backlink pointers always match what is saved
          in their pt_regs instance's ->gpr[1] slot and that
        - their exception frame size equals STACK_INT_FRAME_SIZE, a value
          uncommonly large for non-exception frames.
      
      However, while these are currently true, relying on them would make
      the reliable stacktrace implementation more sensitive towards future
      changes in the exception entry code. Note that false negatives, i.e.
      not detecting exception frames, would silently break the live patching
      consistency model.
      
      Furthermore, certain other places (diagnostic stacktraces, perf, xmon)
      rely on STACK_FRAME_REGS_MARKER as well.
      
      Make the exception exit code clear the on-stack
      STACK_FRAME_REGS_MARKER for those exceptions running on the "normal"
      kernel stack and returning to kernelspace: because the topmost frame
      is ignored by the reliable stack tracer anyway, returns to userspace
      don't need to take care of clearing the marker.
      
      Furthermore, as I don't have the ability to test this on Book 3E or 32
      bits, limit the change to Book 3S and 64 bits.
      
      Fixes: df78d3f6 ("powerpc/livepatch: Implement reliable stack tracing for the consistency model")
      Reported-by: NJoe Lawrence <joe.lawrence@redhat.com>
      Signed-off-by: NNicolai Stange <nstange@suse.de>
      Signed-off-by: NJoe Lawrence <joe.lawrence@redhat.com>
      Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
      Signed-off-by: NSasha Levin <sashal@kernel.org>
      b52681e6
  3. 03 4月, 2019 1 次提交
  4. 07 8月, 2018 1 次提交
    • M
      powerpc/64s: Add support for software count cache flush · ee13cb24
      Michael Ellerman 提交于
      Some CPU revisions support a mode where the count cache needs to be
      flushed by software on context switch. Additionally some revisions may
      have a hardware accelerated flush, in which case the software flush
      sequence can be shortened.
      
      If we detect the appropriate flag from firmware we patch a branch
      into _switch() which takes us to a count cache flush sequence.
      
      That sequence in turn may be patched to return early if we detect that
      the CPU supports accelerating the flush sequence in hardware.
      
      Add debugfs support for reporting the state of the flush, as well as
      runtime disabling it.
      
      And modify the spectre_v2 sysfs file to report the state of the
      software flush.
      Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
      ee13cb24
  5. 30 7月, 2018 2 次提交
  6. 24 7月, 2018 1 次提交
    • N
      powerpc/64s: make PACA_IRQ_HARD_DIS track MSR[EE] closely · 9b81c021
      Nicholas Piggin 提交于
      When the masked interrupt handler clears MSR[EE] for an interrupt in
      the PACA_IRQ_MUST_HARD_MASK set, it does not set PACA_IRQ_HARD_DIS.
      This makes them get out of synch.
      
      With that taken into account, it's only low level irq manipulation
      (and interrupt entry before reconcile) where they can be out of synch.
      This makes the code less surprising.
      
      It also allows the IRQ replay code to rely on the IRQ_HARD_DIS value
      and not have to mtmsrd again in this case (e.g., for an external
      interrupt that has been masked). The bigger benefit might just be
      that there is not such an element of surprise in these two bits of
      state.
      Signed-off-by: NNicholas Piggin <npiggin@gmail.com>
      Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
      9b81c021
  7. 06 6月, 2018 1 次提交
    • B
      powerpc: Add syscall detection for restartable sequences · 6f37be4b
      Boqun Feng 提交于
      Syscalls are not allowed inside restartable sequences, so add a call to
      rseq_syscall() at the very beginning of system call exiting path for
      CONFIG_DEBUG_RSEQ=y kernel. This could help us to detect whether there
      is a syscall issued inside restartable sequences.
      Signed-off-by: NBoqun Feng <boqun.feng@gmail.com>
      Signed-off-by: NMathieu Desnoyers <mathieu.desnoyers@efficios.com>
      Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
      Cc: Joel Fernandes <joelaf@google.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Catalin Marinas <catalin.marinas@arm.com>
      Cc: Dave Watson <davejwatson@fb.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: Andi Kleen <andi@firstfloor.org>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: "H . Peter Anvin" <hpa@zytor.com>
      Cc: Chris Lameter <cl@linux.com>
      Cc: Russell King <linux@arm.linux.org.uk>
      Cc: Michael Ellerman <mpe@ellerman.id.au>
      Cc: Andrew Hunter <ahh@google.com>
      Cc: Michael Kerrisk <mtk.manpages@gmail.com>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: "Paul E . McKenney" <paulmck@linux.vnet.ibm.com>
      Cc: Paul Turner <pjt@google.com>
      Cc: Josh Triplett <josh@joshtriplett.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Ben Maurer <bmaurer@fb.com>
      Cc: linux-api@vger.kernel.org
      Cc: linuxppc-dev@lists.ozlabs.org
      Cc: Andy Lutomirski <luto@amacapital.net>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Link: https://lkml.kernel.org/r/20180602124408.8430-10-mathieu.desnoyers@efficios.com
      6f37be4b
  8. 03 6月, 2018 2 次提交
    • M
      powerpc/64: Use barrier_nospec in syscall entry · 51973a81
      Michael Ellerman 提交于
      Our syscall entry is done in assembly so patch in an explicit
      barrier_nospec.
      
      Based on a patch by Michal Suchanek.
      Signed-off-by: NMichal Suchanek <msuchanek@suse.de>
      Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
      51973a81
    • A
      powerpc/mm/hash: Add missing isync prior to kernel stack SLB switch · 91d06971
      Aneesh Kumar K.V 提交于
      Currently we do not have an isync, or any other context synchronizing
      instruction prior to the slbie/slbmte in _switch() that updates the
      SLB entry for the kernel stack.
      
      However that is not correct as outlined in the ISA.
      
      From Power ISA Version 3.0B, Book III, Chapter 11, page 1133:
      
        "Changing the contents of ... the contents of SLB entries ... can
         have the side effect of altering the context in which data
         addresses and instruction addresses are interpreted, and in which
         instructions are executed and data accesses are performed.
         ...
         These side effects need not occur in program order, and therefore
         may require explicit synchronization by software.
         ...
         The synchronizing instruction before the context-altering
         instruction ensures that all instructions up to and including that
         synchronizing instruction are fetched and executed in the context
         that existed before the alteration."
      
      And page 1136:
      
        "For data accesses, the context synchronizing instruction before the
         slbie, slbieg, slbia, slbmte, tlbie, or tlbiel instruction ensures
         that all preceding instructions that access data storage have
         completed to a point at which they have reported all exceptions
         they will cause."
      
      We're not aware of any bugs caused by this, but it should be fixed
      regardless.
      
      Add the missing isync when updating kernel stack SLB entry.
      
      Cc: stable@vger.kernel.org
      Signed-off-by: NAneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
      [mpe: Flesh out change log with more ISA text & explanation]
      Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
      91d06971
  9. 31 3月, 2018 1 次提交
  10. 19 1月, 2018 5 次提交
  11. 17 1月, 2018 1 次提交
  12. 10 1月, 2018 3 次提交
  13. 11 12月, 2017 1 次提交
  14. 06 11月, 2017 1 次提交
    • M
      powerpc/64s: Replace CONFIG_PPC_STD_MMU_64 with CONFIG_PPC_BOOK3S_64 · 4e003747
      Michael Ellerman 提交于
      CONFIG_PPC_STD_MMU_64 indicates support for the "standard" powerpc MMU
      on 64-bit CPUs. The "standard" MMU refers to the hash page table MMU
      found in "server" processors, from IBM mainly.
      
      Currently CONFIG_PPC_STD_MMU_64 is == CONFIG_PPC_BOOK3S_64. While it's
      annoying to have two symbols that always have the same value, it's not
      quite annoying enough to bother removing one.
      
      However with the arrival of Power9, we now have the situation where
      CONFIG_PPC_STD_MMU_64 is enabled, but the kernel is running using the
      Radix MMU - *not* the "standard" MMU. So it is now actively confusing
      to use it, because it implies that code is disabled or inactive when
      the Radix MMU is in use, however that is not necessarily true.
      
      So s/CONFIG_PPC_STD_MMU_64/CONFIG_PPC_BOOK3S_64/, and do some minor
      formatting updates of some of the affected lines.
      
      This will be a pain for backports, but c'est la vie.
      Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
      4e003747
  15. 31 8月, 2017 1 次提交
  16. 23 8月, 2017 2 次提交
  17. 07 8月, 2017 1 次提交
    • M
      Revert "powerpc/64: Avoid restore_math call if possible in syscall exit" · 44a12806
      Michael Ellerman 提交于
      This reverts commit bc4f65e4.
      
      As reported by Andreas, this commit is causing unrecoverable SLB misses in the
      system call exit path:
      
        Unrecoverable exception 4100 at c00000000000a1ec
        Oops: Unrecoverable exception, sig: 6 [#1]
        SMP NR_CPUS=2 PowerMac
        ...
        CPU: 0 PID: 18626 Comm: rm Not tainted 4.13.0-rc3 #1
        task: c00000018335e080 task.stack: c000000139e50000
        NIP: c00000000000a1ec LR: c00000000000a118 CTR: 0000000000000000
        REGS: c000000139e53bb0 TRAP: 4100   Not tainted  (4.13.0-rc3)
        MSR: 9000000000001030 <SF,HV,ME,IR,DR> CR: 24000044  XER: 20000000 SOFTE: 1
        GPR00: 0000000000000000 c000000139e53e30 c000000000abb500 fffffffffffffffe
        GPR04: c0000001eb866298 0000000000000000 0000000000000000 c00000018335e080
        GPR08: 900000000000d032 0000000000000000 0000000000000002 fffffffffffff001
        GPR12: c000000139e50000 c00000000ffff000 00003fffa8c0dca0 00003fffa8c0dc88
        GPR16: 0000000010000000 0000000000000001 00003fffa8c0eaa0 0000000000000000
        GPR20: 00003fffa8c27528 00003fffa8c27b00 0000000000000000 0000000000000000
        GPR24: 00003fffa8c0d918 00003ffff1b3efa0 00003fffa8c26d68 0000000000000000
        GPR28: 00003fffa8c249e8 00003fffa8c263d0 00003fffa8c27550 00003ffff1b3ef10
        NIP [c00000000000a1ec] system_call_exit+0xc0/0x21c
        LR [c00000000000a118] system_call+0x58/0x6c
        Call Trace:
        [c000000139e53e30] [c00000000000a118] system_call+0x58/0x6c (unreliable)
        Instruction dump:
        64a51000 7c6300d0 f8a101a0 4bffff9c 3c000000 60000006 780007c6 64000000
        60000000 7c004039 4082001c e8ed0170 <88070b78> 88c70b79 7c003214 2c200000
      
      This is caused by us trying to load THREAD_LOAD_FP with MSR_RI=0, and taking an
      SLB miss on the thread struct.
      Reported-by: NAndreas Schwab <schwab@linux-m68k.org>
      Diagnosed-by: NNicholas Piggin <npiggin@gmail.com>
      Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
      44a12806
  18. 03 7月, 2017 5 次提交
  19. 15 6月, 2017 5 次提交
  20. 27 4月, 2017 1 次提交
  21. 24 4月, 2017 1 次提交
  22. 23 4月, 2017 1 次提交
  23. 18 4月, 2017 1 次提交
    • R
      powerpc/kprobe: Fix oops when kprobed on 'stdu' instruction · 9e1ba4f2
      Ravi Bangoria 提交于
      If we set a kprobe on a 'stdu' instruction on powerpc64, we see a kernel
      OOPS:
      
        Bad kernel stack pointer cd93c840 at c000000000009868
        Oops: Bad kernel stack pointer, sig: 6 [#1]
        ...
        GPR00: c000001fcd93cb30 00000000cd93c840 c0000000015c5e00 00000000cd93c840
        ...
        NIP [c000000000009868] resume_kernel+0x2c/0x58
        LR [c000000000006208] program_check_common+0x108/0x180
      
      On a 64-bit system when the user probes on a 'stdu' instruction, the kernel does
      not emulate actual store in emulate_step() because it may corrupt the exception
      frame. So the kernel does the actual store operation in exception return code
      i.e. resume_kernel().
      
      resume_kernel() loads the saved stack pointer from memory using lwz, which only
      loads the low 32-bits of the address, causing the kernel crash.
      
      Fix this by loading the 64-bit value instead.
      
      Fixes: be96f633 ("powerpc: Split out instruction analysis part of emulate_step()")
      Cc: stable@vger.kernel.org # v3.18+
      Signed-off-by: NRavi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
      Reviewed-by: NNaveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
      Reviewed-by: NAnanth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
      [mpe: Change log massage, add stable tag]
      Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>
      9e1ba4f2