1. 02 9月, 2020 3 次提交
    • M
      arm64: Relax ICC_PMR_EL1 accesses when ICC_CTLR_EL1.PMHE is clear · 625b8a72
      Marc Zyngier 提交于
      task #25552995
      
      commit f226650494c6aa87526d12135b7de8b8c074f3de upstream.
      
      The GICv3 architecture specification is incredibly misleading when it
      comes to PMR and the requirement for a DSB. It turns out that this DSB
      is only required if the CPU interface sends an Upstream Control
      message to the redistributor in order to update the RD's view of PMR.
      
      This message is only sent when ICC_CTLR_EL1.PMHE is set, which isn't
      the case in Linux. It can still be set from EL3, so some special care
      is required. But the upshot is that in the (hopefuly large) majority
      of the cases, we can drop the DSB altogether.
      
      This relies on a new static key being set if the boot CPU has PMHE
      set. The drawback is that this static key has to be exported to
      modules.
      
      Cc: Will Deacon <will@kernel.org>
      Cc: James Morse <james.morse@arm.com>
      Cc: Julien Thierry <julien.thierry.kdev@gmail.com>
      Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
      Signed-off-by: NMarc Zyngier <maz@kernel.org>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      Signed-off-by: NZou Cao <zoucao@linux.alibaba.com>
      Reviewed-by: Nluanshi <zhangliguang@linux.alibaba.com>
      625b8a72
    • J
      arm64: Fix incorrect irqflag restore for priority masking · d53b2738
      Julien Thierry 提交于
      task #25552995
      
      commit bd82d4bd21880b7c4d5f5756be435095d6ae07b5 upstream.
      
      When using IRQ priority masking to disable interrupts, in order to deal
      with the PSR.I state, local_irq_save() would convert the I bit into a
      PMR value (GIC_PRIO_IRQOFF). This resulted in local_irq_restore()
      potentially modifying the value of PMR in undesired location due to the
      state of PSR.I upon flag saving [1].
      
      In an attempt to solve this issue in a less hackish manner, introduce
      a bit (GIC_PRIO_IGNORE_PMR) for the PMR values that can represent
      whether PSR.I is being used to disable interrupts, in which case it
      takes precedence of the status of interrupt masking via PMR.
      
      GIC_PRIO_PSR_I_SET is chosen such that (<pmr_value> |
      GIC_PRIO_PSR_I_SET) does not mask more interrupts than <pmr_value> as
      some sections (e.g. arch_cpu_idle(), interrupt acknowledge path)
      requires PMR not to mask interrupts that could be signaled to the
      CPU when using only PSR.I.
      
      [1] https://www.spinics.net/lists/arm-kernel/msg716956.html
      
      Fixes: 4a503217ce37 ("arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking")
      Cc: <stable@vger.kernel.org> # 5.1.x-
      Reported-by: NZenghui Yu <yuzenghui@huawei.com>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Wei Li <liwei391@huawei.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: Christoffer Dall <christoffer.dall@arm.com>
      Cc: James Morse <james.morse@arm.com>
      Cc: Suzuki K Pouloze <suzuki.poulose@arm.com>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Reviewed-by: NMarc Zyngier <marc.zyngier@arm.com>
      Signed-off-by: NJulien Thierry <julien.thierry@arm.com>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      Signed-off-by: NZou Cao <zoucao@linux.alibaba.com>
      Reviewed-by: Nluanshi <zhangliguang@linux.alibaba.com>
      d53b2738
    • J
      arm64: kvm: Unmask PMR before entering guest · 28692731
      Julien Thierry 提交于
      task #25552995
      
      commit 85738e05dc38a80921e1e1944e5b835f6668fc30 upstream
      
      Interrupts masked by ICC_PMR_EL1 will not be signaled to the CPU. This
      means that hypervisor will not receive masked interrupts while running a
      guest.
      
      We need to make sure that all maskable interrupts are masked from the
      time we call local_irq_disable() in the main run loop, and remain so
      until we call local_irq_enable() after returning from the guest, and we
      need to ensure that we see no interrupts at all (including pseudo-NMIs)
      in the middle of the VM world-switch, while at the same time we need to
      ensure we exit the guest when there are interrupts for the host.
      
      We can accomplish this with pseudo-NMIs enabled by:
        (1) local_irq_disable: set the priority mask
        (2) enter guest: set PSTATE.I
        (3)              clear the priority mask
        (4) eret to guest
        (5) exit guest:  set the priotiy mask
                         clear PSTATE.I (and restore other host PSTATE bits)
        (6) local_irq_enable: clear the priority mask.
      Signed-off-by: NJulien Thierry <julien.thierry@arm.com>
      Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
      Reviewed-by: NMarc Zyngier <marc.zyngier@arm.com>
      Reviewed-by: NChristoffer Dall <christoffer.dall@arm.com>
      Cc: Christoffer Dall <christoffer.dall@arm.com>
      Cc: Marc Zyngier <marc.zyngier@arm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: kvmarm@lists.cs.columbia.edu
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      Signed-off-by: NZou Cao <zoucao@linux.alibaba.com>
      Reviewed-by: Nluanshi <zhangliguang@linux.alibaba.com>
      28692731
  2. 18 3月, 2020 1 次提交
  3. 24 3月, 2019 1 次提交
    • J
      KVM: arm64: Forbid kprobing of the VHE world-switch code · 459058f0
      James Morse 提交于
      [ Upstream commit 7d82602909ed9c73b34ad26f05d10db4850a4f8c ]
      
      On systems with VHE the kernel and KVM's world-switch code run at the
      same exception level. Code that is only used on a VHE system does not
      need to be annotated as __hyp_text as it can reside anywhere in the
      kernel text.
      
      __hyp_text was also used to prevent kprobes from patching breakpoint
      instructions into this region, as this code runs at a different
      exception level. While this is no longer true with VHE, KVM still
      switches VBAR_EL1, meaning a kprobe's breakpoint executed in the
      world-switch code will cause a hyp-panic.
      
      echo "p:weasel sysreg_save_guest_state_vhe" > /sys/kernel/debug/tracing/kprobe_events
      echo 1 > /sys/kernel/debug/tracing/events/kprobes/weasel/enable
      lkvm run -k /boot/Image --console serial -p "console=ttyS0 earlycon=uart,mmio,0x3f8"
      
        # lkvm run -k /boot/Image -m 384 -c 3 --name guest-1474
        Info: Placing fdt at 0x8fe00000 - 0x8fffffff
        Info: virtio-mmio.devices=0x200@0x10000:36
      
        Info: virtio-mmio.devices=0x200@0x10200:37
      
        Info: virtio-mmio.devices=0x200@0x10400:38
      
      [  614.178186] Kernel panic - not syncing: HYP panic:
      [  614.178186] PS:404003c9 PC:ffff0000100d70e0 ESR:f2000004
      [  614.178186] FAR:0000000080080000 HPFAR:0000000000800800 PAR:1d00007edbadc0de
      [  614.178186] VCPU:00000000f8de32f1
      [  614.178383] CPU: 2 PID: 1482 Comm: kvm-vcpu-0 Not tainted 5.0.0-rc2 #10799
      [  614.178446] Call trace:
      [  614.178480]  dump_backtrace+0x0/0x148
      [  614.178567]  show_stack+0x24/0x30
      [  614.178658]  dump_stack+0x90/0xb4
      [  614.178710]  panic+0x13c/0x2d8
      [  614.178793]  hyp_panic+0xac/0xd8
      [  614.178880]  kvm_vcpu_run_vhe+0x9c/0xe0
      [  614.178958]  kvm_arch_vcpu_ioctl_run+0x454/0x798
      [  614.179038]  kvm_vcpu_ioctl+0x360/0x898
      [  614.179087]  do_vfs_ioctl+0xc4/0x858
      [  614.179174]  ksys_ioctl+0x84/0xb8
      [  614.179261]  __arm64_sys_ioctl+0x28/0x38
      [  614.179348]  el0_svc_common+0x94/0x108
      [  614.179401]  el0_svc_handler+0x38/0x78
      [  614.179487]  el0_svc+0x8/0xc
      [  614.179558] SMP: stopping secondary CPUs
      [  614.179661] Kernel Offset: disabled
      [  614.179695] CPU features: 0x003,2a80aa38
      [  614.179758] Memory Limit: none
      [  614.179858] ---[ end Kernel panic - not syncing: HYP panic:
      [  614.179858] PS:404003c9 PC:ffff0000100d70e0 ESR:f2000004
      [  614.179858] FAR:0000000080080000 HPFAR:0000000000800800 PAR:1d00007edbadc0de
      [  614.179858] VCPU:00000000f8de32f1 ]---
      
      Annotate the VHE world-switch functions that aren't marked
      __hyp_text using NOKPROBE_SYMBOL().
      Signed-off-by: NJames Morse <james.morse@arm.com>
      Fixes: 3f5c90b8 ("KVM: arm64: Introduce VHE-specific kvm_vcpu_run")
      Acked-by: NMasami Hiramatsu <mhiramat@kernel.org>
      Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
      Signed-off-by: NSasha Levin <sashal@kernel.org>
      459058f0
  4. 23 1月, 2019 1 次提交
  5. 07 9月, 2018 1 次提交
  6. 01 6月, 2018 1 次提交
  7. 25 5月, 2018 6 次提交
    • D
      KVM: arm64: Invoke FPSIMD context switch trap from C · cf412b00
      Dave Martin 提交于
      The conversion of the FPSIMD context switch trap code to C has added
      some overhead to calling it, due to the need to save registers that
      the procedure call standard defines as caller-saved.
      
      So, perhaps it is no longer worth invoking this trap handler quite
      so early.
      
      Instead, we can invoke it from fixup_guest_exit(), with little
      likelihood of increasing the overhead much further.
      
      As a convenience, this patch gives __hyp_switch_fpsimd() the same
      return semantics fixup_guest_exit().  For now there is no
      possibility of a spurious FPSIMD trap, so the function always
      returns true, but this allows it to be tail-called with a single
      return statement.
      Signed-off-by: NDave Martin <Dave.Martin@arm.com>
      Reviewed-by: NMarc Zyngier <marc.zyngier@arm.com>
      Reviewed-by: NChristoffer Dall <christoffer.dall@arm.com>
      Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
      Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
      cf412b00
    • D
      KVM: arm64: Fold redundant exit code checks out of fixup_guest_exit() · 7846b311
      Dave Martin 提交于
      The entire tail of fixup_guest_exit() is contained in if statements
      of the form if (x && *exit_code == ARM_EXCEPTION_TRAP).  As a result,
      we can check just once and bail out of the function early, allowing
      the remaining if conditions to be simplified.
      
      The only awkward case is where *exit_code is changed to
      ARM_EXCEPTION_EL1_SERROR in the case of an illegal GICv2 CPU
      interface access: in that case, the GICv3 trap handling code is
      skipped using a goto.  This avoids pointlessly evaluating the
      static branch check for the GICv3 case, even though we can't have
      vgic_v2_cpuif_trap and vgic_v3_cpuif_trap true simultaneously
      unless we have a GICv3 and GICv2 on the host: that sounds stupid,
      but I haven't satisfied myself that it can't happen.
      
      No functional change.
      Signed-off-by: NDave Martin <Dave.Martin@arm.com>
      Reviewed-by: NMarc Zyngier <marc.zyngier@arm.com>
      Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
      Acked-by: NChristoffer Dall <christoffer.dall@arm.com>
      Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
      7846b311
    • D
      KVM: arm64: Remove redundant *exit_code changes in fpsimd_guest_exit() · ba4f4cb0
      Dave Martin 提交于
      In fixup_guest_exit(), there are a couple of cases where after
      checking what the exit code was, we assign it explicitly with the
      value it already had.
      
      Assuming this is not indicative of a bug, these assignments are not
      needed.
      
      This patch removes the redundant assignments, and simplifies some
      if-nesting that becomes trivial as a result.
      
      No functional change.
      Signed-off-by: NDave Martin <Dave.Martin@arm.com>
      Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
      Acked-by: NMarc Zyngier <marc.zyngier@arm.com>
      Acked-by: NChristoffer Dall <christoffer.dall@arm.com>
      Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
      ba4f4cb0
    • D
      KVM: arm64: Save host SVE context as appropriate · 85acda3b
      Dave Martin 提交于
      This patch adds SVE context saving to the hyp FPSIMD context switch
      path.  This means that it is no longer necessary to save the host
      SVE state in advance of entering the guest, when in use.
      
      In order to avoid adding pointless complexity to the code, VHE is
      assumed if SVE is in use.  VHE is an architectural prerequisite for
      SVE, so there is no good reason to turn CONFIG_ARM64_VHE off in
      kernels that support both SVE and KVM.
      
      Historically, software models exist that can expose the
      architecturally invalid configuration of SVE without VHE, so if
      this situation is detected at kvm_init() time then KVM will be
      disabled.
      Signed-off-by: NDave Martin <Dave.Martin@arm.com>
      Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
      Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
      Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
      85acda3b
    • D
      KVM: arm64: Optimise FPSIMD handling to reduce guest/host thrashing · e6b673b7
      Dave Martin 提交于
      This patch refactors KVM to align the host and guest FPSIMD
      save/restore logic with each other for arm64.  This reduces the
      number of redundant save/restore operations that must occur, and
      reduces the common-case IRQ blackout time during guest exit storms
      by saving the host state lazily and optimising away the need to
      restore the host state before returning to the run loop.
      
      Four hooks are defined in order to enable this:
      
       * kvm_arch_vcpu_run_map_fp():
         Called on PID change to map necessary bits of current to Hyp.
      
       * kvm_arch_vcpu_load_fp():
         Set up FP/SIMD for entering the KVM run loop (parse as
         "vcpu_load fp").
      
       * kvm_arch_vcpu_ctxsync_fp():
         Get FP/SIMD into a safe state for re-enabling interrupts after a
         guest exit back to the run loop.
      
         For arm64 specifically, this involves updating the host kernel's
         FPSIMD context tracking metadata so that kernel-mode NEON use
         will cause the vcpu's FPSIMD state to be saved back correctly
         into the vcpu struct.  This must be done before re-enabling
         interrupts because kernel-mode NEON may be used by softirqs.
      
       * kvm_arch_vcpu_put_fp():
         Save guest FP/SIMD state back to memory and dissociate from the
         CPU ("vcpu_put fp").
      
      Also, the arm64 FPSIMD context switch code is updated to enable it
      to save back FPSIMD state for a vcpu, not just current.  A few
      helpers drive this:
      
       * fpsimd_bind_state_to_cpu(struct user_fpsimd_state *fp):
         mark this CPU as having context fp (which may belong to a vcpu)
         currently loaded in its registers.  This is the non-task
         equivalent of the static function fpsimd_bind_to_cpu() in
         fpsimd.c.
      
       * task_fpsimd_save():
         exported to allow KVM to save the guest's FPSIMD state back to
         memory on exit from the run loop.
      
       * fpsimd_flush_state():
         invalidate any context's FPSIMD state that is currently loaded.
         Used to disassociate the vcpu from the CPU regs on run loop exit.
      
      These changes allow the run loop to enable interrupts (and thus
      softirqs that may use kernel-mode NEON) without having to save the
      guest's FPSIMD state eagerly.
      
      Some new vcpu_arch fields are added to make all this work.  Because
      host FPSIMD state can now be saved back directly into current's
      thread_struct as appropriate, host_cpu_context is no longer used
      for preserving the FPSIMD state.  However, it is still needed for
      preserving other things such as the host's system registers.  To
      avoid ABI churn, the redundant storage space in host_cpu_context is
      not removed for now.
      
      arch/arm is not addressed by this patch and continues to use its
      current save/restore logic.  It could provide implementations of
      the helpers later if desired.
      Signed-off-by: NDave Martin <Dave.Martin@arm.com>
      Reviewed-by: NMarc Zyngier <marc.zyngier@arm.com>
      Reviewed-by: NChristoffer Dall <christoffer.dall@arm.com>
      Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
      Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
      Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
      e6b673b7
    • D
      KVM: arm64: Convert lazy FPSIMD context switch trap to C · ceda9fff
      Dave Martin 提交于
      To make the lazy FPSIMD context switch trap code easier to hack on,
      this patch converts it to C.
      
      This is not amazingly efficient, but the trap should typically only
      be taken once per host context switch.
      Signed-off-by: NDave Martin <Dave.Martin@arm.com>
      Reviewed-by: NMarc Zyngier <marc.zyngier@arm.com>
      Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
      Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
      ceda9fff
  8. 12 4月, 2018 1 次提交
  9. 28 3月, 2018 1 次提交
  10. 20 3月, 2018 1 次提交
  11. 19 3月, 2018 22 次提交
  12. 26 2月, 2018 1 次提交