BXT supports EDP. However since GVT-g only simulate DP monitor
to guest and handles EDP_PSR_IMR and EDP_PSR_IIR as default MMIO
r/w. If guest r/w these IMR/IIR, GVT-g won't simulate the real
HW behavior and below warning is printed:
--------
Interrupt register 0x64838 is not zero: 0xffffffff
WARNING: CPU: 1 PID: 1 at drivers/gpu/drm/i915/i915_irq.c:161
gen3_assert_iir_is_zero+0x34/0xa0

Call Trace:
gen8_de_irq_postinstall+0xad/0x330
gen8_irq_postinstall+0x23/0x80
drm_irq_install+0xb5/0x130
i915_driver_load+0xafd/0xf70
--------
Since GVT-g won't simulate EDP to guest, always set EDP_PSR_IMR
and EDP_PSR_IIR IMR/IIR to 0.
Signed-off-by: NColin Xu <colin.xu@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

The register RENDER_HWS_PGA_GEN7 is renamed to GEN8_GAMW_ECO_DEV_RW_IA
from GEN8 which can control IPS enabling.

v3: MMIO control for IPS is not removed from gen9 but gen10 (Matthew Auld)
v2: IPS of all engines must be enabled together for gen9.
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Leverage most SKL/KBL mmio init info and add different mmio to
BXT specific function init_bxt_mmio_info().
Signed-off-by: NColin Xu <colin.xu@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Broxton belongs to GEN9 family so add to SKL and GEN9 plus.
Signed-off-by: NColin Xu <colin.xu@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

The patch set splits out 2 small locks from the original big gvt lock:
  - vgpu_lock protects per-vGPU data and logic, especially the vGPU
    trap emulation path.
  - sched_lock protects gvt scheudler structure, context schedule logic
    and vGPU's schedule data.

Use vgpu_lock to replace the gvt big lock. By doing this, the
mmio read/write trap path, vgpu virtual event emulation and other
vgpu related process, would be protected under per vgpu_lock.

v9:
  - Change commit author since the patches are improved a lot compared
    with original version.
    Original author: Pei Zhang <pei.zhang@intel.com>
  - Rebase to latest gvt-staging.
v8:
  - Correct coding and comment style.
  - Rebase to latest gvt-staging.
v7:
  - Remove gtt_lock since already proteced by gvt_lock and vgpu_lock.
  - Fix a typo in intel_gvt_deactivate_vgpu, unlock the wrong lock.
v6:
  - Rebase to latest gvt-staging.
v5:
  - Rebase to latest gvt-staging.
  - intel_vgpu_page_track_handler should use vgpu_lock.
v4:
  - Rebase to latest gvt-staging.
  - Protect vgpu->active access with vgpu_lock.
  - Do not wait gpu idle in vgpu_lock.
v3: update to latest code base
v2: add gvt->lock in function gvt_check_vblank_emulation

Performance comparison on Kabylake platform.
  - Configuration:
    Host: Ubuntu 16.04.
    Guest 1 & 2: Ubuntu 16.04.

glmark2 score comparison:
  - Configuration:
    Host: glxgears.
    Guests: glmark2.
+--------------------------------+-----------------+
| Setup                          | glmark2 score   |
+--------------------------------+-----------------+
| unified lock, iommu=on         | 58~62 (avg. 60) |
+--------------------------------+-----------------+
| unified lock, iommu=igfx_off   | 57~61 (avg. 59) |
+--------------------------------+-----------------+
| per-logic lock, iommu=on       | 60~68 (avg. 64) |
+--------------------------------+-----------------+
| per-logic lock, iommu=igfx_off | 61~67 (avg. 64) |
+--------------------------------+-----------------+

lock_stat comparison:
  - Configuration:
    Stop lock stat immediately after boot up.
    Boot 2 VM Guests.
    Run glmark2 in guests.
    Start perf lock_stat for 20 seconds and stop again.
  - Legend: c - contentions; w - waittime-avg
+------------+-----------------+-----------+---------------+------------+
|            | gvt_lock        |sched_lock | vgpu_lock     | gtt_lock   |
+ lock type; +-----------------+-----------+---------------+------------+
| iommu set  | c     | w       | c  | w    | c    | w      | c   | w    |
+------------+-------+---------+----+------+------+--------+-----+------+
| unified;   | 20697 | 839     |N/A | N/A  | N/A  | N/A    | N/A | N/A  |
| on         |       |         |    |      |      |        |     |      |
+------------+-------+---------+----+------+------+--------+-----+------+
| unified;   | 21838 | 658.15  |N/A | N/A  | N/A  | N/A    | N/A | N/A  |
| igfx_off   |       |         |    |      |      |        |     |      |
+------------+-------+---------+----+------+------+--------+-----+------+
| per-logic; | 1553  | 1599.96 |9458|429.97| 5846 | 274.33 | 0   | 0.00 |
| on         |       |         |    |      |      |        |     |      |
+------------+-------+---------+----+------+------+--------+-----+------+
| per-logic; | 1911  | 1678.32 |8335|445.16| 5451 | 244.80 | 0   | 0.00 |
| igfx_off   |       |         |    |      |      |        |     |      |
+------------+-------+---------+----+------+------+--------+-----+------+
Signed-off-by: NPei Zhang <pei.zhang@intel.com>
Signed-off-by: NColin Xu <colin.xu@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Guest OS driver uses PV info registers to deliver cursor hotspot info
to host. This patch is used to get cursor hotspot info from virtual
registers and deliver it to host userspace.

v4->v5:
- remove CI warning.

v3->v4:
- return UINT_MAX when x_hot/y_hot is invalid. (Zhenyu)
- correct version.

v2->v3:
- add validate_hotspot(). (Zhenyu)

v1->v2:
- name as cursor_x_hot/cursor_y_hot. (Zhenyu)
- use i915_reg_t definition instead of magic numbers. (Zhenyu)
Signed-off-by: NTina Zhang <tina.zhang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Return error will cause vm hang and enter failsafe mode.
However, we don't want that happen on detecting an wrong force_to_nonpriv
register write.
Therefore, we just omit the wrong write or patch it to default value.

v2: only return 0 on detecting lri write of registers outside whitelist,
but still return error on other error conditions.  (zhenyu wang)
Signed-off-by: NZhao Yan <yan.y.zhao@intel.com>
Reviewed-by: NZhang Yulei <yulei.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Each ring has a NOPID register and currently they are regarded as default
value of force_to_nonpriv registers in guest drivers
Signed-off-by: NZhao Yan <yan.y.zhao@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

The disable_warn_untrack never prevent gvt from printing untracked
mmio errors. We were disturbed by this error storm and the fix is
just adding them to the list with no essential new change.

This message is only useful for enabling new platform during
developing process. So lower the message level to debug and then
remove disable_warn_untrack.
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

In preparation to enabling -Wimplicit-fallthrough, mark switch cases
where we are expecting to fall through.

Addresses-Coverity-ID: 1466154 ("Missing break in switch")
Signed-off-by: NGustavo A. R. Silva <gustavo@embeddedor.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

In preparation to enabling -Wimplicit-fallthrough, mark switch cases
where we are expecting to fall through.

Addresses-Coverity-ID: 1466154 ("Missing break in switch")
Signed-off-by: NGustavo A. R. Silva <gustavo@embeddedor.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Reduntant message prints when:
  - linux guest creating.
  - dma-buf win10 guest boot.
  - xonotic stress testing in linux guest.

Add below registers to default MMIO handler:
    0xd00, RPM_CONFIG0
    0xd40, RC6_LOCATION
    0x65010, HSW_AUD_MISC_CTRL
    0x6671c,
    0x700a0, CUR_FBC_CTL
    0x7239c,

v2:
  - Should init i915_reg_t using uint32_t instead of _MMIO macro.
    (compiling errors)
  - Use defined offset in i915_reg.h
    (zhenyu)
Signed-off-by: NColin Xu <colin.xu@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Fix below check error by using proper failure message output.

drivers/gpu/drm/i915//gvt/handlers.c:1392 hws_pga_write() error: 'vgpu' dereferencing possible ERR_PTR()
drivers/gpu/drm/i915//gvt/handlers.c:1402 hws_pga_write() error: 'vgpu' dereferencing possible ERR_PTR()
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Fix below warning:

drivers/gpu/drm/i915//gvt/handlers.c:323 gdrst_mmio_write() warn: inconsistent indenting
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Fix below error with minor code refactor.

CHECK   drivers/gpu/drm/i915//gvt/handlers.c
drivers/gpu/drm/i915//gvt/handlers.c:203 sanitize_fence_mmio_access() error: 'vgpu' dereferencing possible ERR_PTR()
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Factor out these two interfaces so we can kill some duplicated code in
scheduler.c.

v2:
  - rename to intel_vgpu_{get,put}_ppgtt_mm
  - refine handle_g2v_notification
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This is a big one and the GVT shadow graphic memory management code is
heavily refined. The new code is more straightforward with less code.

The struct intel_vgpu_mm is restructured to be clearly defined, use
accurate names and some of the original fields are removed which are
really redundant.

Now we only manage ppgtt mm object with mm->ppgtt_mm.lru_list. No need
to mix ppgtt and ggtt together, since one vGPU only has one ggtt object.

v4: Don't invoke ppgtt_free_all_shadow_page before intel_vgpu_destroy_all_ppgtt_mm.
v3: Add GVT_RING_CTX_NR_PDPS to avoid confusing about the PDPs.
v2: Split some changes into small standalone patches.
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Only reset vgpu execlist state of the exact engine which gets reset
request from VM. After read context status from HWSP enabled, KMD will use
the saved CSB read pointer but not always read from MMIO. When one engine
reset happen, only the read pointer of this engine will be reset, in GVT-g
host side also need to align with this policy, otherwise VM may get wrong
CSB status after one engine reset compeleted.

v2: Split refine and fix patch, code refine(Zhenyu)
v3: Move active flag of vgpu scheduler into sched_data(Zhenyu)

Cc: Fred Gao <fred.gao@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>

Using per engine ops will be more flexible, here refine sub-ops(init,
clean) as per engine operation align with reset operation. This change also
will be used in next fix patch for VM engine reset.

Cc: Fred Gao <fred.gao@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>

Running 4.15 Linux kernel in VM will cause host GVT reports
'untrack mmio 0x701a0' errror, which identifies the PLANE_KEYMAX
registers. Add them to track list.

v2: rebase to latest staging code.
Signed-off-by: NPei Zhang <pei.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>

We had previous hack that tried to accept either i915_reg_t or offset
value to access vGPU virtual/shadow regs which broke that purpose to
be type safe in context. This one trys to explicitly separate the usage
of typed mmio reg with real offset.

Old vgpu_vreg(offset) helper is used only for offset now with new
vgpu_vreg_t(reg) is used for i915_reg_t only. Convert left usage
of that to new helper.

Also fixed left KASAN warning issues caused by previous hack.

v2: rebase, fixup against recent mmio switch change
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Always requires properly defined i915_reg_t type for MMIO handler
definition.

Fix kasan warning of "drivers/gpu/drm/i915/gvt/handlers.c:2397:1: error: the frame size of 32120 bytes is larger than 8192 bytes"
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

mmio_read_from_hw() let vgpu could read hw reg, if vgpu's workload
is running on hw, things is good. Otherwise vgpu will get other
vgpu's reg val, it is unsafe.

This patch limit such hw access to active vgpu. If vgpu isn't
running on hw, the reg read of this vgpu will get the last active
val which saved at schedule_out.

v2: ring timestamp is walking continuously even if the ring is idle.
    so read hw directly. (Zhenyu)
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
(cherry picked from commit 295764cd)

Since many emulation logic needs to convert the offset of ring registers
into ring id, we export it for other caller which might need it.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
(cherry picked from commit 62a6a537)

The ring_id maximum boundary is being compared using the > operator
instead of >=, leading to an off-by-one error and an out of bounds
write into array vgpu->hws_pga[].  Fix this by simply using the
correct comparison operator. Also re-work another comparison that
uses the comparison > I915_NUM_ENGINES - 1 to use the >= idiom using
to keep this consistent in this code.

Detected by CoverityScan, CID#1462404 ("Out-of-bounds write")

Fixes: a2ae95af ("drm/i915/gvt: update CSB and CSB write pointer in virtual HWSP")
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

The switch statement is missing breaks for the cases of
GVT_FAILSAFE_INSUFFICIENT_RESOURCE and GVT_FAILSAFE_GUEST_ERR. Add them
in.

Detected by CoverityScan, CID#1462416 ("Missing break in switch")

Fixes: e011c6ce ("drm/i915/gvt: Add VM healthy check for workload_thread")
Fixes: a33fc7a0 ("drm/i915/gvt: enter failsafe mode when guest requires more resources")
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

With different settings of compressed data hash mode between VMs and host
may cause gpu issues.

Commit: 1999f108 ("drm/i915/gvt: Disable compression workaround for Gen9")
disable compression workaround of guest in gvt host to align with host.

Commit: 93564044 ("drm/i915: Switch over to the LLC/eLLC hotspot avoidance
hash mode for CCS") add compression workaround, then we can remove the
skl_misc_ctl_write hanlder.

Better solution should be always keeping same settings as host, and bypass
the write request from VMs, but it need to fetch data from host's
"Context".

Cc: Zhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

mmio_read_from_hw() let vgpu could read hw reg, if vgpu's workload
is running on hw, things is good. Otherwise vgpu will get other
vgpu's reg val, it is unsafe.

This patch limit such hw access to active vgpu. If vgpu isn't
running on hw, the reg read of this vgpu will get the last active
val which saved at schedule_out.

v2: ring timestamp is walking continuously even if the ring is idle.
    so read hw directly. (Zhenyu)
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This patch add a function intel_gvt_for_each_tracked_mmio() to
iterate each tracked mmio. The caller don't be aware of how the
tracked mmios are presented internally.

v2: remove snapshot_hw_mmio_registers().
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

The engine provides a mirror of the CSB and CSB write pointer in the HWSP.
Read these status from virtual HWSP in VM can reduce CPU utilization while
applications have much more short GPU workloads. Here we update the
corresponding data in virtual HWSP as it in virtual MMIO.

Before read these status from HWSP in GVT-g VM, please ensure the host
support it by checking the BIT(3) of caps in PVINFO.

Virtual HWSP only support GEN8+ platform, since the HWSP MMIO may change
follow the platform update, please add the corresponding MMIO emulation
when enable new platforms in GVT-g.

v3 : Add address audit in HWSP address update.

v4 :
     Separate this patch with enalbe virtual HWSP in VM.
     Use intel_gvt_render_mmio_to_ring_id() to determine ring_id by offset.

v5 : Remove unnessary check about Gen8, GVT-g only support Gen8+.
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Since many emulation logic needs to convert the offset of ring registers
into ring id, we export it for other caller which might need it.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

We need debugfs entry to expose some debug information of gvt and vGPUs.
The first tool will be added is mmio-diff, which help to find the
difference values of host and vGPU mmio. It's useful for platform
enabling.

This patch just add a basic debugfs infrastructure, each vGPU has its own
sub-folder. Two simple attributes are created as a template.
.
├── num_tracked_mmio
├── vgpu1
|   └── active
└── vgpu2
    └── active
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

Introduce vGPU submission ops to support easy switching submission mode
of one vGPU between different OSes.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

When a scan error occurs in dispatch_workload, this patch is to
check the healthy state and free all the queued workloads before
the failsafe mode is entered.
Signed-off-by: Nfred gao <fred.gao@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Move tlb_handle_pending into intel_vgpu_submssion since it belongs to a
part of vGPU submission stuffs
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

Introduce intel_vgpu_submission to hold all members related to submission
in struct intel_vgpu before.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

The context descriptors in elsp_dwords are stored in a reversed order and
the definition of context descriptor is also reversed. The revesred stuff
is hard to be used and might cause misunderstanding. Make them in the right
oder for following code re-factoring.

Tested on my SKL NUC.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

When a workload is too heavy to finish it in gpu hang check timer
intervals(1.5), gpu hang check function will check ACTHD register
value to decide whether gpu is real dead or not. On real hw,
ACTHD is updated by HW when workload is running, then host kernel
won't think it is gpu hang. while guest kernel always read a constant
ACTHD value as GVT doesn't supply ACTHD emulate handler, then
guest kernel detects a fake gpu hang.

To remove such guest fake gpu hang, this patch supply ACTHD
mmio read handler which read real HW ACTHD register directly.
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/b4c9a097-3e62-124e-6856-b0c37764df7b@intel.com

The mmio read handler for ring timestmap / instdone register are same
as reading hw value directly.

Extract it as common function to reduce code duplications.
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

Inspect if the host has VCS2 ring by host i915 macro in MMIO_RING_F().
Also this helps on reducing some LOCs.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>