Move tlb_handle_pending into intel_vgpu_submssion since it belongs to a
part of vGPU submission stuffs
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

Introduce intel_vgpu_submission to hold all members related to submission
in struct intel_vgpu before.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

To move workload related functions into scheduler.c, an expected way is
to collect all the init/clean functions related to vGPU workload
submission into fewer functions.

Rename intel_vgpu_{init, clean}_gvt_context() for above usage in future.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

Guest i915 full ppgtt functionality was blocking by an issue, which would
lead to gpu hardware hang. Guest i915 driver may update the ppgtt table
just before this workload is going to be submitted to the hardware by
device model. This case wasn't handled well by device model before, due
to the small time window between removing old ppgtt entry and adding the
new one. Errors occur when the workload is executed by hardware during
that small time window. This patch is to remove this time window by adding
the new ppgtt entry first and then remove the old one.

Changes in v2:
- Move VGT_CAPS_FULL_PPGTT introduction to patch 2/4. (Joonas)

Changes since v2:
- Divide the whole patch set into two separate patch series, with one
  patch in i915 side to check guest i915 full ppgtt capability and enable
  it when this capability is supported by the device model, and the other
  one in gvt side which fixs the blocking issue and enables the device
  model to provide the capability to guest. And this patch focuses on gvt
  side. (Joonas)
- Change the title from "reorder the shadow ppgtt update process by adding
  entry first" to "Fix guest i915 full ppgtt blocking issue". (Tina)

Changes since v3:
- Rebase to the latest branch.

Changes since v4:
- Tested by Tina Zhang.

Changes since v5:
- Rebase to the latest branch.

v6:
- Update full 48bit ppgtt definition

Cc: Tina Zhang <tina.zhang@intel.com>
Signed-off-by: NTina Zhang <tina.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

When doing the VGPU reset, we don't need to do the gtt/ppgtt reset.
This will make the GVT to do the ppgtt shadow every time for
a workload and caused really bad performance after a VGPU reset.
This patch will make sure ppgtt clean only happen at device module
level reset to fix this.
Signed-off-by: NChuanxiao Dong <chuanxiao.dong@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Use resetting_eng to identify which engine is resetting
so the rest ones' workload won't be impacted

v2:
- use ENGINE_MASK(ring_id) instead of (1 << ring_id). (Zhenyu)
Signed-off-by: NChuanxiao Dong <chuanxiao.dong@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

during the emulation of virtual reset:
1. only reset the engine related mmio ending with MMIO
   offset Master_IRQ, not include display stuff.

2. fences are not required to set default
   value as well to prevent screen flicking.

this will fix the issue of Guest screen hang while running
Force tdr in Linux guest.

v2:
- only reset the engine related mmio. (Zhenyu & Zhiyuan)
v3:
- IMR/Ring mode registers are not save/restored. (Changbin)
v4:
- redefine the MMIO reset offset for easy understanding. (Zhenyu)
- pvinfo can be reset. (Zhenyu)
v5:
- add more comments for mmio reset. (Zhenyu)

Cc: Changbin Du <changbin.du@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Lv zhiyuan <zhiyuan.lv@intel.com>
Cc: Zhang Yulei <yulei.zhang@intel.com>
Signed-off-by: Nfred gao <fred.gao@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

From commit d1a513be ("drm/i915/gvt: add resolution definition for vGPU
type"), small type has been restricted to small resolution, so not
require larger high GM size any more. Change to smaller 384M for more
VM creation with vGPU enabled which still perform reasonable workload.

Fixes: d1a513be ("drm/i915/gvt: add resolution definition for vGPU type")
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
(cherry picked from commit bf39ec33)
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

This patch introduces two functions for activating/de-activating vGPU in
mdev ops.

A racing condition was found between virtual vblank emulation and KVGMT
mdev release path. V-blank emulation will emulate and inject V-blank
interrupt for every active vGPU with holding gvt->lock, while in mdev
release path, it will directly release hypervisor handle without changing
vGPU status or taking gvt->lock, so a kernel oops is encountered when
vblank emulation is injecting a interrupt with a invalid hypervisor
handle. (Reported by Terrence)

To solve this problem, we factor out vGPU activation/de-activation from
vGPU creation/destruction path and let KVMGT mdev release ops de-activate
the vGPU before release hypervisor handle. Once a vGPU is de-activated,
GVT-g will not emulate v-blank for it or touch the hypervisor handle.

Fixes: 659643f7 ("drm/i915/gvt/kvmgt: add vfio/mdev support to KVMGT")
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

vGPU resource is allocated by scheduler. To account for non-allocated
free cycles, we create an idle vGPU as the placeholder similar to idle task
concept, which is useful to handle some corner cases in scheduling policy.
Signed-off-by: NPing Gao <ping.a.gao@intel.com>
Reviewed-by: NKevin Tian <kevin.tian@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

The weight defines proportional control of physical GPU resource
shared between vGPUs. So far the weight is tied to a specific vGPU
type, i.e when creating multiple vGPUs with different types, they
will inherit different weights.

e.g. The weight of type GVTg_V5_2 is 8, the weight of type GVTg_V5_4
is 4, so vGPU of type GVTg_V5_2 has double vGPU resource of vGPU type
GVTg_V5_4.

TODO: allow user control the weight setting in the future.
Signed-off-by: NPing Gao <ping.a.gao@intel.com>
Reviewed-by: NKevin Tian <kevin.tian@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

From commit d1a513be ("drm/i915/gvt: add resolution definition for vGPU
type"), small type has been restricted to small resolution, so not
require larger high GM size any more. Change to smaller 384M for more
VM creation with vGPU enabled which still perform reasonable workload.

Fixes: d1a513be ("drm/i915/gvt: add resolution definition for vGPU type")
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This assigns resolution definition for each vGPU type. For smaller
resource type we should limit max resolution, so e.g limit to 1024x768
for 64M type, others are still default to 1920x1200.

v2: Fix for actual 1920x1200 resolution
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Previous vGPU type create tried to determine vGPU type name e.g _1, _2
based on the number of mdev devices can be created, but different type
might have very different resource size depending on physical device.
We need to split type name vs. actual mdev resource and create fixed
vGPU type with determined size for consistence.

With this we'd like to fix vGPU types for _1, _2, _4 and _8 now, each
type has fixed defined resource size. Available mdev instances that could
be created is determined by physical resource, and user should query
for that before creating.

Cc: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: NKevin Tian <kevin.tian@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

New failsafe mode is introduced, when we detect guest not supporting
GVT-g.
In failsafe mode, we will ignore all the MMIO and cfg space read/write
from guest.

This patch can fix the issue that when guest kernel or graphics driver
version is too low, there will be a lot of kernel traces in host.

V5: rebased onto latest gvt-staging
V4: changed coding style by Zhenyu and Ping's advice
V3: modified coding style and error messages according to Zhenyu's comment
V2: 1) implemented MMIO/GTT/WP pages read/write logic; 2) used a unified
function to enter failsafe mode
Signed-off-by: NMin He <min.he@intel.com>
Signed-off-by: NPei Zhang <pei.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

We also need reset vGPU virtual display emulation. Since all vreg has
been cleared, we need reset display related vreg to reflect our display
setting.
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Cc: Ping Gao <ping.a.gao@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

As now gvt init after knowing hw resource info, we can determine vGPU
type from machine size instead of pre-defined value.
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Our function tests found several issues related to reusing vGPU
instance. They are qemu reboot failure, guest tdr after reboot, host
hang when reboot guest. All these issues are caused by dirty status
inherited from last VM.

This patch fix all these issues by resetting a virtual GPU before VM
use it. The reset logical is put into a low level function
_intel_gvt_reset_vgpu(), which supports Device Model Level Reset, Full
GT Reset and Per-Engine Reset.

vGPU Device Model Level Reset (DMLR) simulates the PCI reset to reset
the whole vGPU to default state as when it is created, including GTT,
execlist, scratch pages, cfg space, mmio space, pvinfo page, scheduler
and fence registers. The ultimate goal of vGPU DMLR is that reuse a
vGPU instance by different virtual machines. When we reassign a vGPU
to a virtual machine we must issue such reset first.

Full GT Reset and Per-Engine GT Reset are soft reset flow for GPU engines
(Render, Blitter, Video, Video Enhancement). It is defined by GPU Spec.
Unlike the FLR, GT reset only reset particular resource of a vGPU per
the reset request. Guest driver can issue a GT reset by programming
the virtual GDRST register to reset specific virtual GPU engine or all
engines.

Since vGPU DMLR and GT reset can share some code so we implement both
these two into one single function intel_gvt_reset_vgpu_locked(). The
parameter dmlr is to identify if we will do FLR or GT reset. The
parameter engine_mask is to specific the engines that need to be
resetted. If value ALL_ENGINES is given for engine_mask, it means
the caller requests a full gt reset that we will reset all virtual
GPU engines.
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Reviewed-by: NJike Song <jike.song@intel.com>
Reviewed-by: NKevin Tian <kevin.tian@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Move the mmio space inititation function setup_vgpu_mmio()
and cleanup function clean_vgpu_mmio() in vgpu.c to dedicated
source file mmio.c, and rename them as intel_vgpu_init_mmio()
and intel_vgpu_clean_mmio() respectively.
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Move the configuration space inititation function setup_vgpu_cfg_space()
in vgpu.c to dedicated source file cfg_space.c, and rename the function
as intel_vgpu_init_cfg_space().
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Once idr_alloc gets called data is allocated within the idr list, if
any error occurs afterwards, we should undo that by idr_remove on the
error path.
Signed-off-by: NJike Song <jike.song@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Previous high mem size initialized for vGPU type was too small which caused
failure for some VMs. This trys to take minimal value of 384MB for each VM and
enlarge default high mem size to make guest driver happy.
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Initiate param.primary to 1. We should be primary currently.
Signed-off-by: NDu, Changbin <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Since there's no opregion in vgpu so clear the opregion bits in case
guest access it.
Signed-off-by: NXiaoguang Chen <xiaoguang.chen@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Gvt gdrst handler handle_device_reset() invoke function
setup_vgpu_mmio() to reset mmio status. In this case,
the virtual mmio memory has been allocated already. The
new allocation just cause old mmio memory leakage.
Signed-off-by: NDu, Changbin <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

There are currently 4 methods in intel_gvt_io_emulation_ops
to emulate CFG/MMIO reading/writing for intel vGPU. A possibly
better scope is: add 3 more methods for vgpu create/destroy/reset
respectively, and rename the ops to 'intel_gvt_ops', then pass
it to the MPT module (say the future kvmgt) to use: they are
all methods for external usage.
Signed-off-by: NJike Song <jike.song@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Current GVT contains some obsolete logic originally cooked to
support the old, non-vfio kvmgt, which is actually workarounds.
We don't support that anymore, so it's safe to remove it and
make a better framework.
Signed-off-by: NJike Song <jike.song@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

By providing predefined vGPU types, users can choose which type a vgpu
to create and use, without specifying detailed parameters.
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NJike Song <jike.song@intel.com>

Full vGPU reset need to release all the shadow PPGGT pages to avoid
unnecessary write-protect and also should re-initialize pvinfo after
resetting vregs to keep pvinfo correct.
Signed-off-by: NPing Gao <ping.a.gao@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

i915 core should only call functions and structures exposed through
intel_gvt.h. Remove internal gvt.h and i915_pvinfo.h.

Change for internal intel_gvt structure as private handler which
not requires to expose gvt internal structure for i915 core.

v2: Fix per Chris's comment
- carefully handle dev_priv->gvt assignment
- add necessary bracket for macro helper
- forward declartion struct intel_gvt
- keep free operation within same file handling alloc

v3: fix use after free and remove intel_gvt.initialized

v4: change to_gvt() to an inline
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

As different VM may configure different render MMIOs when executing
workload, to schedule workloads between different VM, the render MMIOs
have to be switched.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This patch introduces a vGPU schedule policy framework, with a timer based
schedule policy module for now
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This patch introduces the vGPU workload scheduler routines.

GVT workload scheduler is responsible for picking and executing GVT workload
from current scheduled vGPU. Before the workload is submitted to host i915,
the guest execlist context will be shadowed in the host GVT shadow context.
the instructions in guest ring buffer will be copied into GVT shadow ring
buffer. Then GVT-g workload scheduler will scan the instructions in guest
ring buffer and submit it to host i915.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This patch introduces the vGPU workload submission logics.

Under virtualization environment, guest will submit workload through
virtual execlist submit port. The submitted workload load will be wrapped
into an gvt workload which will be picked by GVT workload scheduler and
executed on host i915 later.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This patch introduces the vGPU execlist virtualization.

Under virtulization environment, HW execlist interface are fully emulated
including virtual CSB emulation, virtual execlist emulation. The framework
will emulate the virtual CSB according to the guest workload running status
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This patch introduces the GVT-g display virtualization.

It consists a collection of display MMIO handlers, like power well register
handler, pipe register handler, plane register handler, which will emulate
all display MMIOs behavior to support virtual mode setting sequence for
guest.
Signed-off-by: NBing Niu <bing.niu@intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This patch introduces the generic vGPU MMIO emulation intercept
framework.  The MPT modules will request GVT-g core logic to
emulate MMIO read/write through IO emulation operations
callback when hypervisor trapped a guest GTTMMIO read/write.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This patch introduces vGPU PCI configuration space virtualization.

- Adjust the trapped GPFN(Guest Page Frame Number) window of virtual GEN
PCI BAR 0 when guest initializes PCI BAR 0 address.

- Emulate OpRegion when guest touches OpRegion.

- Pass-through a part of aperture to guest when guest initializes
aperture BAR.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

The vGPU graphics memory emulation framework is responsible for graphics
memory table virtualization. Under virtualization environment, a VM will
populate the page table entry with guest page frame number(GPFN/GFN), while
HW needs a page table filled with MFN(Machine frame number). The
relationship between GFN and MFN(Machine frame number) is managed by
hypervisor, while GEN HW doesn't have such knowledge to translate a GFN.

To solve this gap, shadow GGTT/PPGTT page table is introdcued.

For GGTT, the GFN inside the guest GGTT page table entry will be translated
into MFN and written into physical GTT MMIO registers when guest write
virtual GTT MMIO registers.

For PPGTT, a shadow PPGTT page table will be created and write-protected
translated from guest PPGTT page table.  And the shadow page table root
pointers will be written into the shadow context after a guest workload
is shadowed.

vGPU graphics memory emulation framework consists:

- Per-GEN HW platform page table entry bits extract/de-extract routines.
- GTT MMIO register emulation handlers, which will call hypercall to do
GFN->MFN translation when guest write GTT MMIO register
- PPGTT shadow page table routines, e.g. shadow create/destroy/out-of-sync
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

A vGPU represents a virtual Intel GEN hardware, which consists following
virtual resources:

- Configuration space (virtualized)
- HW registers (virtualized)
- GGTT memory space (partitioned)
- GPU page table (shadowed)
- Fence registers (partitioned)

* virtualized: fully emulated by GVT-g.
* partitioned: Only a part of the HW resource is allowed to be accessed
by VM.
* shadowed: Resource needs to be translated and shadowed before getting
applied into HW.

This patch introduces vGPU life cycle management framework, which is
responsible for creating/destroying a vGPU and preparing/free resources
related to a vGPU.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>