Return error will cause vm hang and enter failsafe mode.
However, we don't want that happen on detecting an wrong force_to_nonpriv
register write.
Therefore, we just omit the wrong write or patch it to default value.

v2: only return 0 on detecting lri write of registers outside whitelist,
but still return error on other error conditions.  (zhenyu wang)
Signed-off-by: NZhao Yan <yan.y.zhao@intel.com>
Reviewed-by: NZhang Yulei <yulei.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Each ring has a NOPID register and currently they are regarded as default
value of force_to_nonpriv registers in guest drivers
Signed-off-by: NZhao Yan <yan.y.zhao@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

The disable_warn_untrack never prevent gvt from printing untracked
mmio errors. We were disturbed by this error storm and the fix is
just adding them to the list with no essential new change.

This message is only useful for enabling new platform during
developing process. So lower the message level to debug and then
remove disable_warn_untrack.
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

In preparation to enabling -Wimplicit-fallthrough, mark switch cases
where we are expecting to fall through.

Addresses-Coverity-ID: 1466154 ("Missing break in switch")
Signed-off-by: NGustavo A. R. Silva <gustavo@embeddedor.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

The assertion for len is wrong, so fix it. And for where to validate
user input, we should not warn by call trace.

[ 290.584739] WARNING: CPU: 0 PID: 1471 at drivers/gpu/drm/i915/gvt/handlers.c:969 dp_aux_ch_ctl_mmio_write+0x394/0x430 [i915]
[ 290.586113] task: ffff880111fe8000 task.stack: ffffc90044a9c000
[ 290.586192] RIP: e030:dp_aux_ch_ctl_mmio_write+0x394/0x430 [i915]
[ 290.586258] RSP: e02b:ffffc90044a9fd88 EFLAGS: 00010282
[ 290.586315] RAX: 0000000000000017 RBX: 0000000000000003 RCX: ffffffff82461148
[ 290.586391] RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000201
[ 290.586468] RBP: ffffc90043ed1000 R08: 0000000000000248 R09: 00000000000003d8
[ 290.586544] R10: ffffc90044bdd314 R11: 0000000000000011 R12: 0000000000064310
[ 290.586621] R13: 00000000fe4003ff R14: ffffc900432d1008 R15: ffff88010fa7cb40
[ 290.586701] FS: 0000000000000000(0000) GS:ffff880123200000(0000) knlGS:0000000000000000
[ 290.586787] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 290.586849] CR2: 00007f67ea44e000 CR3: 0000000116078000 CR4: 0000000000042660
[ 290.586926] Call Trace:
[ 290.586958] ? __switch_to_asm+0x40/0x70
[ 290.587017] intel_vgpu_mmio_reg_rw+0x1ec/0x3c0 [i915]
[ 290.587087] intel_vgpu_emulate_mmio_write+0xa8/0x2c0 [i915]
[ 290.587151] xengt_emulation_thread+0x501/0x7a0 [xengt]
[ 290.587208] ? __schedule+0x3c6/0x890
[ 290.587250] ? wait_woken+0x80/0x80
[ 290.587290] kthread+0xfc/0x130
[ 290.587326] ? xengt_gpa_to_va+0x1f0/0x1f0 [xengt]
[ 290.587378] ? kthread_create_on_node+0x70/0x70
[ 290.587429] ? do_group_exit+0x3a/0xa0
[ 290.587471] ret_from_fork+0x35/0x40

Fixes: 04d348ae ("drm/i915/gvt: vGPU display virtualization")
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

In preparation to enabling -Wimplicit-fallthrough, mark switch cases
where we are expecting to fall through.

Addresses-Coverity-ID: 1466154 ("Missing break in switch")
Signed-off-by: NGustavo A. R. Silva <gustavo@embeddedor.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Reduntant message prints when:
  - linux guest creating.
  - dma-buf win10 guest boot.
  - xonotic stress testing in linux guest.

Add below registers to default MMIO handler:
    0xd00, RPM_CONFIG0
    0xd40, RC6_LOCATION
    0x65010, HSW_AUD_MISC_CTRL
    0x6671c,
    0x700a0, CUR_FBC_CTL
    0x7239c,

v2:
  - Should init i915_reg_t using uint32_t instead of _MMIO macro.
    (compiling errors)
  - Use defined offset in i915_reg.h
    (zhenyu)
Signed-off-by: NColin Xu <colin.xu@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Fix below check error by using proper failure message output.

drivers/gpu/drm/i915//gvt/handlers.c:1392 hws_pga_write() error: 'vgpu' dereferencing possible ERR_PTR()
drivers/gpu/drm/i915//gvt/handlers.c:1402 hws_pga_write() error: 'vgpu' dereferencing possible ERR_PTR()
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Fix below warning:

drivers/gpu/drm/i915//gvt/handlers.c:323 gdrst_mmio_write() warn: inconsistent indenting
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Fix below error with minor code refactor.

CHECK   drivers/gpu/drm/i915//gvt/handlers.c
drivers/gpu/drm/i915//gvt/handlers.c:203 sanitize_fence_mmio_access() error: 'vgpu' dereferencing possible ERR_PTR()
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Factor out these two interfaces so we can kill some duplicated code in
scheduler.c.

v2:
  - rename to intel_vgpu_{get,put}_ppgtt_mm
  - refine handle_g2v_notification
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This is a big one and the GVT shadow graphic memory management code is
heavily refined. The new code is more straightforward with less code.

The struct intel_vgpu_mm is restructured to be clearly defined, use
accurate names and some of the original fields are removed which are
really redundant.

Now we only manage ppgtt mm object with mm->ppgtt_mm.lru_list. No need
to mix ppgtt and ggtt together, since one vGPU only has one ggtt object.

v4: Don't invoke ppgtt_free_all_shadow_page before intel_vgpu_destroy_all_ppgtt_mm.
v3: Add GVT_RING_CTX_NR_PDPS to avoid confusing about the PDPs.
v2: Split some changes into small standalone patches.
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Only reset vgpu execlist state of the exact engine which gets reset
request from VM. After read context status from HWSP enabled, KMD will use
the saved CSB read pointer but not always read from MMIO. When one engine
reset happen, only the read pointer of this engine will be reset, in GVT-g
host side also need to align with this policy, otherwise VM may get wrong
CSB status after one engine reset compeleted.

v2: Split refine and fix patch, code refine(Zhenyu)
v3: Move active flag of vgpu scheduler into sched_data(Zhenyu)

Cc: Fred Gao <fred.gao@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>

Using per engine ops will be more flexible, here refine sub-ops(init,
clean) as per engine operation align with reset operation. This change also
will be used in next fix patch for VM engine reset.

Cc: Fred Gao <fred.gao@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>

Running 4.15 Linux kernel in VM will cause host GVT reports
'untrack mmio 0x701a0' errror, which identifies the PLANE_KEYMAX
registers. Add them to track list.

v2: rebase to latest staging code.
Signed-off-by: NPei Zhang <pei.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>

We had previous hack that tried to accept either i915_reg_t or offset
value to access vGPU virtual/shadow regs which broke that purpose to
be type safe in context. This one trys to explicitly separate the usage
of typed mmio reg with real offset.

Old vgpu_vreg(offset) helper is used only for offset now with new
vgpu_vreg_t(reg) is used for i915_reg_t only. Convert left usage
of that to new helper.

Also fixed left KASAN warning issues caused by previous hack.

v2: rebase, fixup against recent mmio switch change
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Always requires properly defined i915_reg_t type for MMIO handler
definition.

Fix kasan warning of "drivers/gpu/drm/i915/gvt/handlers.c:2397:1: error: the frame size of 32120 bytes is larger than 8192 bytes"
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

mmio_read_from_hw() let vgpu could read hw reg, if vgpu's workload
is running on hw, things is good. Otherwise vgpu will get other
vgpu's reg val, it is unsafe.

This patch limit such hw access to active vgpu. If vgpu isn't
running on hw, the reg read of this vgpu will get the last active
val which saved at schedule_out.

v2: ring timestamp is walking continuously even if the ring is idle.
    so read hw directly. (Zhenyu)
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
(cherry picked from commit 295764cd)

Since many emulation logic needs to convert the offset of ring registers
into ring id, we export it for other caller which might need it.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
(cherry picked from commit 62a6a537)

The ring_id maximum boundary is being compared using the > operator
instead of >=, leading to an off-by-one error and an out of bounds
write into array vgpu->hws_pga[].  Fix this by simply using the
correct comparison operator. Also re-work another comparison that
uses the comparison > I915_NUM_ENGINES - 1 to use the >= idiom using
to keep this consistent in this code.

Detected by CoverityScan, CID#1462404 ("Out-of-bounds write")

Fixes: a2ae95af ("drm/i915/gvt: update CSB and CSB write pointer in virtual HWSP")
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

The switch statement is missing breaks for the cases of
GVT_FAILSAFE_INSUFFICIENT_RESOURCE and GVT_FAILSAFE_GUEST_ERR. Add them
in.

Detected by CoverityScan, CID#1462416 ("Missing break in switch")

Fixes: e011c6ce ("drm/i915/gvt: Add VM healthy check for workload_thread")
Fixes: a33fc7a0 ("drm/i915/gvt: enter failsafe mode when guest requires more resources")
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

With different settings of compressed data hash mode between VMs and host
may cause gpu issues.

Commit: 1999f108 ("drm/i915/gvt: Disable compression workaround for Gen9")
disable compression workaround of guest in gvt host to align with host.

Commit: 93564044 ("drm/i915: Switch over to the LLC/eLLC hotspot avoidance
hash mode for CCS") add compression workaround, then we can remove the
skl_misc_ctl_write hanlder.

Better solution should be always keeping same settings as host, and bypass
the write request from VMs, but it need to fetch data from host's
"Context".

Cc: Zhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

mmio_read_from_hw() let vgpu could read hw reg, if vgpu's workload
is running on hw, things is good. Otherwise vgpu will get other
vgpu's reg val, it is unsafe.

This patch limit such hw access to active vgpu. If vgpu isn't
running on hw, the reg read of this vgpu will get the last active
val which saved at schedule_out.

v2: ring timestamp is walking continuously even if the ring is idle.
    so read hw directly. (Zhenyu)
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This patch add a function intel_gvt_for_each_tracked_mmio() to
iterate each tracked mmio. The caller don't be aware of how the
tracked mmios are presented internally.

v2: remove snapshot_hw_mmio_registers().
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

The engine provides a mirror of the CSB and CSB write pointer in the HWSP.
Read these status from virtual HWSP in VM can reduce CPU utilization while
applications have much more short GPU workloads. Here we update the
corresponding data in virtual HWSP as it in virtual MMIO.

Before read these status from HWSP in GVT-g VM, please ensure the host
support it by checking the BIT(3) of caps in PVINFO.

Virtual HWSP only support GEN8+ platform, since the HWSP MMIO may change
follow the platform update, please add the corresponding MMIO emulation
when enable new platforms in GVT-g.

v3 : Add address audit in HWSP address update.

v4 :
     Separate this patch with enalbe virtual HWSP in VM.
     Use intel_gvt_render_mmio_to_ring_id() to determine ring_id by offset.

v5 : Remove unnessary check about Gen8, GVT-g only support Gen8+.
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Since many emulation logic needs to convert the offset of ring registers
into ring id, we export it for other caller which might need it.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

We need debugfs entry to expose some debug information of gvt and vGPUs.
The first tool will be added is mmio-diff, which help to find the
difference values of host and vGPU mmio. It's useful for platform
enabling.

This patch just add a basic debugfs infrastructure, each vGPU has its own
sub-folder. Two simple attributes are created as a template.
.
├── num_tracked_mmio
├── vgpu1
|   └── active
└── vgpu2
    └── active
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

Introduce vGPU submission ops to support easy switching submission mode
of one vGPU between different OSes.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

When a scan error occurs in dispatch_workload, this patch is to
check the healthy state and free all the queued workloads before
the failsafe mode is entered.
Signed-off-by: Nfred gao <fred.gao@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Move tlb_handle_pending into intel_vgpu_submssion since it belongs to a
part of vGPU submission stuffs
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

Introduce intel_vgpu_submission to hold all members related to submission
in struct intel_vgpu before.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

The context descriptors in elsp_dwords are stored in a reversed order and
the definition of context descriptor is also reversed. The revesred stuff
is hard to be used and might cause misunderstanding. Make them in the right
oder for following code re-factoring.

Tested on my SKL NUC.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

When a workload is too heavy to finish it in gpu hang check timer
intervals(1.5), gpu hang check function will check ACTHD register
value to decide whether gpu is real dead or not. On real hw,
ACTHD is updated by HW when workload is running, then host kernel
won't think it is gpu hang. while guest kernel always read a constant
ACTHD value as GVT doesn't supply ACTHD emulate handler, then
guest kernel detects a fake gpu hang.

To remove such guest fake gpu hang, this patch supply ACTHD
mmio read handler which read real HW ACTHD register directly.
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/b4c9a097-3e62-124e-6856-b0c37764df7b@intel.com

The mmio read handler for ring timestmap / instdone register are same
as reading hw value directly.

Extract it as common function to reduce code duplications.
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

Inspect if the host has VCS2 ring by host i915 macro in MMIO_RING_F().
Also this helps on reducing some LOCs.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

Remove one duplicated MMIO GEN6_PCODE_MAILBOX. Duplicated MMIO will
cause host GVT-g initialization failure.

Fixes: 9c3a16c8 ("drm/i915/hsw+: Add support for multiple power well regs")
Signed-off-by: NJian Jun Chen <jian.jun.chen@intel.com>
Acked-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Future platforms increase the number of power wells which require
additional control registers. A convenient way to select the correct
register is to use the high bits of the power well ID as index. This
patch only prepares for this, while upcoming platform enabling patches
will add the actual new power well IDs and corresponding power well
control registers.

Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Cc: Animesh Manna <animesh.manna@intel.com>
Cc: Rakshmi Bhatia <rakshmi.bhatia@intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NAnimesh Manna <animesh.manna@intel.com>
Reviewed-by: NRakshmi Bhatia <rakshmi.bhatia@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20170814151530.24154-2-imre.deak@intel.com

Remove duplicated MMIO entries in the tracked MMIO list. -EEXIST
is returned if duplicated MMIO entries are found when new MMIO
entry is added.

v2:
- Use WARN(1, ...) for more verbose message. (Zhenyu)
Signed-off-by: NJian Jun Chen <jian.jun.chen@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Changbin Du <changbin.du@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Reviewed-by: NYulei Zhang <yulei.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

When linux guest access mmio with __raw_i915_read64 or __raw_i915_write64,
its length is 8 bytes.

This fix the linux guest in xengt couldn't boot up as it fail in
reading pv_info->magic.

Fixes: 65f9f6fe ("drm/i915/gvt: Optimize MMIO register handling for some large MMIO blocks")
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

MMIO block with tracked mmio, is introduced for the sake of performance
of searching tracked mmio. All the tracked mmio needs to get the initial
value from the HW state during vGPU being created. This patch is to
initialize the tracked registers in MMIO block with the HW state.

v2: Add "Fixes:" line for this patch (Zhenyu)

Fixes: 65f9f6fe ("drm/i915/gvt: Optimize MMIO register handling for some large MMIO blocks")
Signed-off-by: NTina Zhang <tina.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>