In commit 6bb2a2af ("drm/i915/gvt: Fix crash after request->hw_context change"),
forgot to handle workload scan path in ELSP handler case which was to
optimize scanning earlier instead of in gvt submission thread, so request
alloc and add was splitting then which is against right process.

This trys to do a partial revert of that commit which still has workload
request alloc helper and make sure shadow state population is handled after
request alloc for target state buffer.

v3: Fix missed workload status setting in request alloc error path
v2: Fix dispatch workload err path that should add request after alloc anyway.

Fixes: 6bb2a2af ("drm/i915/gvt: Fix crash after request->hw_context change")
Cc: Bin Yang <bin.yang@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Tested-by: NBin Yang <bin.yang@intel.com>
Reviewed-by: NXiaolin Zhang <xiaolin.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Recently gvt shadow ctx create ppgtt table and this ppgtt's root
pointer is modified at workload dispatch, then we lose the original
ppgtt's root pointer, this causes the ppgtt destroy function abnormal
as it will release the wrong root table.

This patch save i915 context ppgtt root pointer at shadow
ctx creation and restore it at shadow ctx destruction.

v2: Split save and restore function (Zhenyu)

Fixes:4f15665c("drm/i915: Add ppgtt to GVT GEM context")
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Currently the guest couldn't boot up under GVT-g environment as the
following call trace exists:
[  272.504762] BUG: unable to handle kernel NULL pointer dereference at 0000000000000100
[  272.504834] Call Trace:
[  272.504852]  execlists_context_pin+0x2b2/0x520 [i915]
[  272.504869]  intel_gvt_scan_and_shadow_workload+0x50/0x4d0 [i915]
[  272.504887]  intel_vgpu_create_workload+0x3e2/0x570 [i915]
[  272.504901]  intel_vgpu_submit_execlist+0xc0/0x2a0 [i915]
[  272.504916]  elsp_mmio_write+0xc7/0x130 [i915]
[  272.504930]  intel_vgpu_mmio_reg_rw+0x24a/0x4c0 [i915]
[  272.504944]  intel_vgpu_emulate_mmio_write+0xac/0x240 [i915]
[  272.504947]  intel_vgpu_rw+0x22d/0x270 [kvmgt]
[  272.504949]  intel_vgpu_write+0x164/0x1f0 [kvmgt]

GVT GEM context is created by i915_gem_context_create_gvt() which
doesn't allocate ppgtt. So GVT GEM context structure doesn't have
a valid i915_hw_ppgtt.

This patch create ppgtt table at GVT GEM context creation, then assign
shadow ppgtt's root table address to this ppgtt when shadow ppgtt will
be used on GPU. So GVT GEM context has valid ppgtt address. But note
that this ppgtt only contain valid ppgtt root table address, the table
entry in this ppgtt structure are invalid.

Fixes:4a3d3f67("drm/i915: Match code to comment and enforce ppgtt for execlists")
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Reviewed-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/1539841231-3157-1-git-send-email-xiong.y.zhang@intel.com

Some workloads may be prepared in vgpu's queue but not be scheduled
to run yet. If vgpu is released at this time, they will not be freed
in workload complete callback and so need to be freed in vgpu release
operation.

Add new vgpu_release operation in gvt_ops to stop vgpu and release
runtime resources. gvt_ops vgpu_deactivate operation will only stop
vgpu.

v2: add new gvt ops to clean vgpu running status (Xiong Zhang)
Signed-off-by: NHang Yuan <hang.yuan@linux.intel.com>
Reviewed-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Caught by W=1 to fix left wrong function comment doc.
Reviewed-by: NHang Yuan <hang.yuan@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

if a context is a restore inhibit context, gfx hw only load the first page
for ring context, so we only need to copy from guest the 1 page too.

v3: use "return" instead of "goto" for inhibit case. (zhenyu wang)
v2: move judgement of restore inhibit to a macro in  mmio_context.h
Signed-off-by: NZhao Yan <yan.y.zhao@intel.com>
Acked-by: NHang Yuan <hang.yuan@linux.intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Handling such a late error in request construction is tricky, but to
accommodate future patches which may allocate here, we potentially could
err. To handle the error after already adjusting global state to track
the new request, we must finish and submit the request. But we don't
want to use the request as not everything is being tracked by it, so we
opt to cancel the commands inside the request.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20180706103947.15919-3-chris@chris-wilson.co.uk

BXT forcewake is handled in the same way as SKL/KBL.

v2: Add missing inhibit_context restore for BXT.
Signed-off-by: NColin Xu <colin.xu@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Array 'pdp_pair' of size 1 may use index value(s) 1..7.
Changed to pdps[8] to avoid confusion.
Signed-off-by: NXinyun Liu <xinyun.liu@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

When we do shadowing, workload's request might not be allocated yet,
so we still require shadow context's object. And when complete workload,
delay to zero workload's request pointer after used for update guest context.

v2: Move request alloc earlier as already try to track shadow status
depending on request state, which also facilitate to use request->hw_context
for target engine context reference.

Fixes: 1fc44d9b ("drm/i915: Store a pointer to intel_context in i915_request")
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Weinan Li <weinan.z.li@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20180521081752.31056-1-zhenyuw@linux.intel.com

To ease the frequent and ugly pointer dance of
&request->gem_context->engine[request->engine->id] during request
submission, store that pointer as request->hw_context. One major
advantage that we will exploit later is that this decouples the logical
context state from the engine itself.

v2: Set mock_context->ops so we don't crash and burn in selftests.
    Cleanups from Tvrtko.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Acked-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20180517212633.24934-3-chris@chris-wilson.co.uk

In the next patch, we want to store the intel_context pointer inside
i915_request, as it is frequently access via a convoluted dance when
submitting the request to hw. Having two context pointers inside
i915_request leads to confusion so first rename the existing
i915_gem_context pointer to i915_request.gem_context.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20180517212633.24934-1-chris@chris-wilson.co.uk

The scheduler lock(gvt->sched_lock) is used to protect gvt
scheduler logic, including the gvt scheduler structure(gvt->scheduler
and per vgpu schedule data(vgpu->sched_data, vgpu->sched_ctl).

v9:
  - Change commit author since the patches are improved a lot compared
    with original version.
    Original author: Pei Zhang <pei.zhang@intel.com>
  - Rebase to latest gvt-staging.
v8:
  - Correct coding wqstyle.
  - Rebase to latest gvt-staging.
v7:
  - Remove gtt_lock since already proteced by gvt_lock and vgpu_lock.
v6:
  - Rebase to latest gvt-staging.
v5:
  - Rebase to latest gvt-staging.
v4:
  - Rebase to latest gvt-staging.
v3: update to latest code base
Signed-off-by: NPei Zhang <pei.zhang@intel.com>
Signed-off-by: NColin Xu <colin.xu@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

The patch set splits out 2 small locks from the original big gvt lock:
  - vgpu_lock protects per-vGPU data and logic, especially the vGPU
    trap emulation path.
  - sched_lock protects gvt scheudler structure, context schedule logic
    and vGPU's schedule data.

Use vgpu_lock to replace the gvt big lock. By doing this, the
mmio read/write trap path, vgpu virtual event emulation and other
vgpu related process, would be protected under per vgpu_lock.

v9:
  - Change commit author since the patches are improved a lot compared
    with original version.
    Original author: Pei Zhang <pei.zhang@intel.com>
  - Rebase to latest gvt-staging.
v8:
  - Correct coding and comment style.
  - Rebase to latest gvt-staging.
v7:
  - Remove gtt_lock since already proteced by gvt_lock and vgpu_lock.
  - Fix a typo in intel_gvt_deactivate_vgpu, unlock the wrong lock.
v6:
  - Rebase to latest gvt-staging.
v5:
  - Rebase to latest gvt-staging.
  - intel_vgpu_page_track_handler should use vgpu_lock.
v4:
  - Rebase to latest gvt-staging.
  - Protect vgpu->active access with vgpu_lock.
  - Do not wait gpu idle in vgpu_lock.
v3: update to latest code base
v2: add gvt->lock in function gvt_check_vblank_emulation

Performance comparison on Kabylake platform.
  - Configuration:
    Host: Ubuntu 16.04.
    Guest 1 & 2: Ubuntu 16.04.

glmark2 score comparison:
  - Configuration:
    Host: glxgears.
    Guests: glmark2.
+--------------------------------+-----------------+
| Setup                          | glmark2 score   |
+--------------------------------+-----------------+
| unified lock, iommu=on         | 58~62 (avg. 60) |
+--------------------------------+-----------------+
| unified lock, iommu=igfx_off   | 57~61 (avg. 59) |
+--------------------------------+-----------------+
| per-logic lock, iommu=on       | 60~68 (avg. 64) |
+--------------------------------+-----------------+
| per-logic lock, iommu=igfx_off | 61~67 (avg. 64) |
+--------------------------------+-----------------+

lock_stat comparison:
  - Configuration:
    Stop lock stat immediately after boot up.
    Boot 2 VM Guests.
    Run glmark2 in guests.
    Start perf lock_stat for 20 seconds and stop again.
  - Legend: c - contentions; w - waittime-avg
+------------+-----------------+-----------+---------------+------------+
|            | gvt_lock        |sched_lock | vgpu_lock     | gtt_lock   |
+ lock type; +-----------------+-----------+---------------+------------+
| iommu set  | c     | w       | c  | w    | c    | w      | c   | w    |
+------------+-------+---------+----+------+------+--------+-----+------+
| unified;   | 20697 | 839     |N/A | N/A  | N/A  | N/A    | N/A | N/A  |
| on         |       |         |    |      |      |        |     |      |
+------------+-------+---------+----+------+------+--------+-----+------+
| unified;   | 21838 | 658.15  |N/A | N/A  | N/A  | N/A    | N/A | N/A  |
| igfx_off   |       |         |    |      |      |        |     |      |
+------------+-------+---------+----+------+------+--------+-----+------+
| per-logic; | 1553  | 1599.96 |9458|429.97| 5846 | 274.33 | 0   | 0.00 |
| on         |       |         |    |      |      |        |     |      |
+------------+-------+---------+----+------+------+--------+-----+------+
| per-logic; | 1911  | 1678.32 |8335|445.16| 5451 | 244.80 | 0   | 0.00 |
| igfx_off   |       |         |    |      |      |        |     |      |
+------------+-------+---------+----+------+------+--------+-----+------+
Signed-off-by: NPei Zhang <pei.zhang@intel.com>
Signed-off-by: NColin Xu <colin.xu@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This reverts commit 11474e90.

There are issues which will block the host preemption before, instead of
disabling it use one workaround "setting max priority for gvt context"
to avoid the gvt context be preempted by the host. Now the issues have been
cleared, so revert this patch to enable host preemption.

v2:
- refine description(Zhenyu)
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

Make life easier in upcoming patches by moving the context_pin and
context_unpin vfuncs into inline helpers.

v2: Fixup mock_engine to mark the context as pinned on use.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20180430131503.5375-2-chris@chris-wilson.co.uk

The checks are misleading and not required [1].

[1] https://lkml.org/lkml/2018/3/19/1792

Addresses-Coverity-ID: 1466017
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NGustavo A. R. Silva <gustavo@embeddedor.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

For perfomance purpose, scanning of non-privileged batch buffer is turned
off by default. But for debugging purpose, it can be turned on via debugfs.
After scanning, we submit the original non-privileged batch buffer into
hardware, so that the scanning is only a peeking window of guest submitted
commands and will not affect the execution results.

v4:
- refine debugfs print format&content (zhenyu wang)
- print engine id instread of engine name to prevent potential memory leak
  in debugfs warning message. (zhenyu wang)

v3:
- change vgpu->scan_nonprivbb from type bool to u32, so it is able to
  selectively turn on/off scanning of non-privileged batch buffer on engine
  level. e.g.
  if vgpu->scan_nonprivbb=3, then it will scan non-privileged batch buffer
  on engine 0 and 1.
- in debugfs interface to set vgpu->scan_nonprivbb, print warning message
  to warn user and explicitly tell state change in kernel log (zhenyu wang)
v2:
- rebase
- update comments for start_gma_offset (henry)
Signed-off-by: NZhao Yan <yan.y.zhao@intel.com>
Reviewed-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

Today we only want to pass along the priority to engine->schedule(), but
in the future we want to have much more control over the various aspects
of the GPU during a context's execution, for example controlling the
frequency allowed. As we need an ever growing number of parameters for
scheduling, move those into a struct for convenience.

v2: Move the anonymous struct into its own function for legibility and
ye olde gcc.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20180418184052.7129-3-chris@chris-wilson.co.uk

Our shadow context content is from guest but with masked control reg like
CTX_CONTEXT_CONTROL, we need to make sure all settings from guest would be set
when this context is on hw, this trys to force mask enable bits for all to
ensure every bits setting would be effective on hw.

One regression found related to once inhibit bit is set, gpu engine are working
on inhibit state until MI_LOAD_REG_IMM command or context image clear inhibit
bit with mask bit set to 1, and val bit set to 0. In gvt-g currently workload
has the highest priority, so gvt-g workload could trigger preempt context
easily, preempt context set inhibit bit, then gvt-g workload is scheduled in,
but gvt-g workload shadow context image usually doesn't set inhibit mask bit,
so gpu is still in inhibit state when gvt workload is running. This caused gpu
hang.
Suggested-by: NZhang, Xiong <xiong.y.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Reviewed-by: NZhang, Xiong <xiong.y.zhang@intel.com>

The PDPs of a shadow page will only be valid after a vGPU mm is pinned.
So the PDPs in the shadow context should be updated then.
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This is to fix warning got as:

[ 6730.476938] ------------[ cut here ]------------
[ 6730.476979] Bad or missing usercopy whitelist? Kernel memory exposure attempt detected from SLAB object 'gvt-g_vgpu_workload' (offset 120, size 4)!
[ 6730.477021] WARNING: CPU: 2 PID: 441 at mm/usercopy.c:81 usercopy_warn+0x7e/0xa0
[ 6730.477042] Modules linked in: tun(E) bridge(E) stp(E) llc(E) kvmgt(E) x86_pkg_temp_thermal(E) vfio_mdev(E) intel_powerclamp(E) mdev(E) coretemp(E) vfio_iommu_type1(E) vfio(E) kvm_intel(E) kvm(E) hid_generic(E) irqbypass(E) crct10dif_pclmul(E) crc32_pclmul(E) usbhid(E) i915(E) crc32c_intel(E) hid(E) ghash_clmulni_intel(E) pcbc(E) aesni_intel(E) aes_x86_64(E) crypto_simd(E) cryptd(E) glue_helper(E) intel_cstate(E) idma64(E) evdev(E) virt_dma(E) iTCO_wdt(E) intel_uncore(E) intel_rapl_perf(E) intel_lpss_pci(E) sg(E) shpchp(E) mei_me(E) pcspkr(E) iTCO_vendor_support(E) intel_lpss(E) intel_pch_thermal(E) prime_numbers(E) mei(E) mfd_core(E) video(E) acpi_pad(E) button(E) binfmt_misc(E) ip_tables(E) x_tables(E) autofs4(E) ext4(E) crc16(E) mbcache(E) jbd2(E) fscrypto(E) sd_mod(E) e1000e(E) xhci_pci(E) sdhci_pci(E)
[ 6730.477244]  ptp(E) cqhci(E) xhci_hcd(E) pps_core(E) sdhci(E) mmc_core(E) i2c_i801(E) usbcore(E) thermal(E) fan(E)
[ 6730.477276] CPU: 2 PID: 441 Comm: gvt workload 0 Tainted: G            E    4.16.0-rc1-gvt-staging-0213+ #127
[ 6730.477303] Hardware name:  /NUC6i5SYB, BIOS SYSKLi35.86A.0039.2016.0316.1747 03/16/2016
[ 6730.477326] RIP: 0010:usercopy_warn+0x7e/0xa0
[ 6730.477340] RSP: 0018:ffffba6301223d18 EFLAGS: 00010286
[ 6730.477355] RAX: 0000000000000000 RBX: ffff8f41caae9838 RCX: 0000000000000006
[ 6730.477375] RDX: 0000000000000007 RSI: 0000000000000082 RDI: ffff8f41dad166f0
[ 6730.477395] RBP: 0000000000000004 R08: 0000000000000576 R09: 0000000000000000
[ 6730.477415] R10: ffffffffb1293fb2 R11: 00000000ffffffff R12: 0000000000000001
[ 6730.477447] R13: ffff8f41caae983c R14: ffff8f41caae9838 R15: 00007f183ca2b000
[ 6730.477467] FS:  0000000000000000(0000) GS:ffff8f41dad00000(0000) knlGS:0000000000000000
[ 6730.477489] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6730.477506] CR2: 0000559462817291 CR3: 000000028b46c006 CR4: 00000000003626e0
[ 6730.477526] Call Trace:
[ 6730.477537]  __check_object_size+0x9c/0x1a0
[ 6730.477562]  __kvm_write_guest_page+0x45/0x90 [kvm]
[ 6730.477585]  kvm_write_guest+0x46/0x80 [kvm]
[ 6730.477599]  kvmgt_rw_gpa+0x9b/0xf0 [kvmgt]
[ 6730.477642]  workload_thread+0xa38/0x1040 [i915]
[ 6730.477659]  ? do_wait_intr_irq+0xc0/0xc0
[ 6730.477673]  ? finish_wait+0x80/0x80
[ 6730.477707]  ? clean_workloads+0x120/0x120 [i915]
[ 6730.477722]  kthread+0x111/0x130
[ 6730.477733]  ? _kthread_create_worker_on_cpu+0x60/0x60
[ 6730.477750]  ? exit_to_usermode_loop+0x6f/0xb0
[ 6730.477766]  ret_from_fork+0x35/0x40
[ 6730.477777] Code: 48 c7 c0 20 e3 25 b1 48 0f 44 c2 41 50 51 41 51 48 89 f9 49 89 f1 4d 89 d8 4c 89 d2 48 89 c6 48 c7 c7 78 e3 25 b1 e8 b2 bc e4 ff <0f> ff 48 83 c4 18 c3 48 c7 c6 09 d0 26 b1 49 89 f1 49 89 f3 eb
[ 6730.477849] ---[ end trace cae869c1c323e45a ]---

By whitelist guest page write from workload struct allocated from kmem cache.
Reviewed-by: NHang Yuan <hang.yuan@linux.intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
(cherry picked from commit 5627705406874df57fdfad3b4e0c9aedd3b007df)

Once the ring buffer is copied to ring_scan_buffer and scanned,
the shadow batch buffer start address is only updated into
ring_scan_buffer, not the real ring address allocated through
intel_ring_begin in later copy_workload_to_ring_buffer.

This patch is only to set the right shadow batch buffer address
from Ring buffer, not include the shadow_wa_ctx.

v2:
- refine some comments. (Zhenyu)
v3:
- fix typo in title. (Zhenyu)
v4:
- remove the unnecessary comments. (Zhenyu)
- add comments in bb_start_cmd_va update. (Zhenyu)

Fixes: 0a53bc07 ("drm/i915/gvt: Separate cmd scan from request allocation")
Cc: stable@vger.kernel.org  # v4.15
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Cc: Yulei Zhang <yulei.zhang@intel.com>
Signed-off-by: Nfred gao <fred.gao@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

When populating shadow ctx from guest, we should handle oa related
registers in hw ctx, so that they will not be overlapped by guest oa
configs. This patch made it possible to capture oa data from host for
both host and guests.
Signed-off-by: NMin He <min.he@intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>

There is one issue relates to Coarse Power Gating(CPG) on KBL NUC in GVT-g,
vgpu can't get the correct default context by updating the registers before
inhibit context submission. It always get back the hardware default value
unless the inhibit context submission happened before the 1st time
forcewake put. With this wrong default context, vgpu will run with
incorrect state and meet unknown issues.

The solution is initialize these mmios by adding lri command in ring buffer
of the inhibit context, then gpu hardware has no chance to go down RC6 when
lri commands are right being executed, and then vgpu can get correct
default context for further use.

v3:
- fix code fault, use 'for' to loop through mmio render list(Zhenyu)

v4:
- save the count of engine mmio need to be restored for inhibit context and
  refine some comments. (Kevin)

v5:
- code rebase

Cc: Kevin Tian <kevin.tian@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Factor out these two interfaces so we can kill some duplicated code in
scheduler.c.

v2:
  - rename to intel_vgpu_{get,put}_ppgtt_mm
  - refine handle_g2v_notification
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Reviewed-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

If we manage an object with a reference count, then its life cycle
must flow the reference count operations. Meanwhile, change the
operation functions to generic name *put* and *get*.
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

This is a big one and the GVT shadow graphic memory management code is
heavily refined. The new code is more straightforward with less code.

The struct intel_vgpu_mm is restructured to be clearly defined, use
accurate names and some of the original fields are removed which are
really redundant.

Now we only manage ppgtt mm object with mm->ppgtt_mm.lru_list. No need
to mix ppgtt and ggtt together, since one vGPU only has one ggtt object.

v4: Don't invoke ppgtt_free_all_shadow_page before intel_vgpu_destroy_all_ppgtt_mm.
v3: Add GVT_RING_CTX_NR_PDPS to avoid confusing about the PDPs.
v2: Split some changes into small standalone patches.
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

We want to de-emphasize the link between the request (dependency,
execution and fence tracking) from GEM and so rename the struct from
drm_i915_gem_request to i915_request. That is we may implement the GEM
user interface on top of requests, but they are an abstraction for
tracking execution rather than an implementation detail of GEM. (Since
they are not tied to HW, we keep the i915 prefix as opposed to intel.)

In short, the spatch:
@@

@@
- struct drm_i915_gem_request
+ struct i915_request

A corollary to contracting the type name, we also harmonise on using
'rq' shorthand for local variables where space if of the essence and
repetition makes 'request' unwieldy. For globals and struct members,
'request' is still much preferred for its clarity.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Michał Winiarski <michal.winiarski@intel.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20180221095636.6649-1-chris@chris-wilson.co.ukReviewed-by: NMika Kuoppala <mika.kuoppala@linux.intel.com>
Reviewed-by: NMichał Winiarski <michal.winiarski@intel.com>
Acked-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>

Only reset vgpu execlist state of the exact engine which gets reset
request from VM. After read context status from HWSP enabled, KMD will use
the saved CSB read pointer but not always read from MMIO. When one engine
reset happen, only the read pointer of this engine will be reset, in GVT-g
host side also need to align with this policy, otherwise VM may get wrong
CSB status after one engine reset compeleted.

v2: Split refine and fix patch, code refine(Zhenyu)
v3: Move active flag of vgpu scheduler into sched_data(Zhenyu)

Cc: Fred Gao <fred.gao@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>

Using per engine ops will be more flexible, here refine sub-ops(init,
clean) as per engine operation align with reset operation. This change also
will be used in next fix patch for VM engine reset.

Cc: Fred Gao <fred.gao@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NZhi Wang <zhi.a.wang@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>

This is to workaround guest driver hang regression after
preemption enable that gvt hasn't enabled handling of that
for guest workload. So in effect this disables preemption
for gvt context now.
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
(cherry picked from commit 1603660b)

We shouldn't mark inactive for vGPU context if preempted,
which would still be re-scheduled later. So keep active state.

Fixes: d6c05113 ("drm/i915/execlists: Distinguish the incomplete context notifies")
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
(cherry picked from commit da5f99ea)

mmio_read_from_hw() let vgpu could read hw reg, if vgpu's workload
is running on hw, things is good. Otherwise vgpu will get other
vgpu's reg val, it is unsafe.

This patch limit such hw access to active vgpu. If vgpu isn't
running on hw, the reg read of this vgpu will get the last active
val which saved at schedule_out.

v2: ring timestamp is walking continuously even if the ring is idle.
    so read hw directly. (Zhenyu)
Signed-off-by: NXiong Zhang <xiong.y.zhang@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
(cherry picked from commit 295764cd)

This is to workaround guest driver hang regression after
preemption enable that gvt hasn't enabled handling of that
for guest workload. So in effect this disables preemption
for gvt context now.
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

We shouldn't mark inactive for vGPU context if preempted,
which would still be re-scheduled later. So keep active state.

Fixes: d6c05113 ("drm/i915/execlists: Distinguish the incomplete context notifies")
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

The current schedule policy rely on a 1ms timer to execute workload. This
can introduce maximum 1ms unnecessary latency. This is especially bad for
small media workloads.

And I don't think we need this timer for QoS, but the change is not simply
remove the code. So I made a new API intel_gvt_kick_schedule() for future
change.
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Convert the macro to a function which should always be preferred.
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>

Previously the performance is improved through the workload auditing
and shadowing ahead of vGPU scheduling, however, there is the case that
more requests are allocated in submit_context before the previous request
is added, the timeline will hold its seqno which is later.

This patch is to move the request alloc to dispatch_workload function,
where is the same place as request is added.

It will fix the issue of kernel BUG for (timeline->seqno != request->fence.seqno)
check when add_request.

Fixes: 89ea20b9 ("drm/i915/gvt: Factor out scan and shadow from workload dispatch")
Signed-off-by: NChuanxiao Dong <chuanxiao.dong@intel.com>
Signed-off-by: Nfred gao <fred.gao@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
(cherry picked from commit f2880e04)

The caller of shadow_context_status_change may disable irqs. So it is not
safe to use spin_unlock_bh in such context. Let's switch to irqsave version
for safety.

------------[ cut here ]------------
WARNING: CPU: 2 PID: 4504 at kernel/softirq.c:161 __local_bh_enable_ip+0x46/0x60
[  168.797710] Hardware name: Dell Inc. OptiPlex 7040/0Y7WYT, BIOS 1.2.8 01/26/2016
[  168.797712] task: ffff8c693d22db80 task.stack: ffffb51b482bc000
[  168.797718] RIP: 0010:__local_bh_enable_ip+0x46/0x60
[  168.797721] RSP: 0018:ffffb51b482bfa10 EFLAGS: 00010046
[  168.797724] RAX: 0000000000000046 RBX: ffff8c6900278000 RCX: 00000000ffffffff
[  168.797726] RDX: 0000000000000001 RSI: 0000000000000200 RDI: ffffffffc06a0330
[  168.797728] RBP: ffffb51b482bfa10 R08: 0000000000000000 R09: ffff8c690027cb90
[  168.797730] R10: ffffb51b482bfa40 R11: 00000004072f0001 R12: 0000000000000000
[  168.797732] R13: 0000000000000000 R14: ffff8c690027ca9c R15: 0000000000000000
[  168.797735] FS:  00007ff187c56700(0000) GS:ffff8c6959d00000(0000) knlGS:0000000000000000
[  168.797738] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  168.797740] CR2: 0000562bc0c3991f CR3: 0000000430614006 CR4: 00000000003606e0
[  168.797742] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  168.797744] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  168.797745] Call Trace:
[  168.797755]  _raw_spin_unlock_bh+0x1e/0x20
[  168.797826]  shadow_context_status_change+0x120/0x1e0 [i915]
[  168.797831]  notifier_call_chain+0x4a/0x70
[  168.797834]  atomic_notifier_call_chain+0x1a/0x20
[  168.797896]  execlists_cancel_port_requests+0x4f/0x80 [i915]
[  168.797956]  reset_common_ring+0x30/0x100 [i915]
[  168.798007]  i915_gem_reset_engine+0x114/0x330 [i915]
[  168.798060]  ? i915_gem_retire_requests+0x75/0x180 [i915]
[  168.798111]  i915_gem_reset+0x3e/0xb0 [i915]
[  168.798149]  i915_reset+0x10b/0x1c0 [i915]
[  168.798187]  i915_reset_device+0x209/0x220 [i915]
[  168.798225]  ? gen8_gt_irq_ack+0x170/0x170 [i915]
[  168.798229]  ? __queue_work+0x430/0x430
[  168.798270]  i915_handle_error+0x285/0x420 [i915]
[  168.798275]  ? mntput+0x24/0x40
[  168.798281]  ? terminate_walk+0x8e/0xf0
[  168.798328]  i915_wedged_set+0x84/0xc0 [i915]
[  168.798333]  simple_attr_write+0xab/0xc0
[  168.798337]  full_proxy_write+0x54/0x90
[  168.798343]  __vfs_write+0x37/0x170
[  168.798349]  ? common_file_perm+0x4c/0x100
[  168.798355]  ? apparmor_file_permission+0x1a/0x20
[  168.798361]  ? security_file_permission+0x3b/0xc0
[  168.798365]  vfs_write+0xb8/0x1b0
[  168.798370]  SyS_write+0x55/0xc0
[  168.798376]  entry_SYSCALL_64_fastpath+0x1e/0xa9

Fixes: 0e86cc9c ("drm/i915/gvt: implement per-vm mmio switching optimization")
Signed-off-by: NChangbin Du <changbin.du@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>