Ring aggregated doorbel to make unmapped queue scheduled in mes firmware.
Signed-off-by: NLe Ma <le.ma@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NJack Xiao <Jack.Xiao@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The scratch register should be accessed through MMIO instead of RLCG
in SRIOV, since it being used in RLCG register access function.

Fixes: d54762cc ("drm/amdgpu: nuke dynamic gfx scratch reg allocation")
Signed-off-by: NZhenGuo Yin <zhenguo.yin@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable pipe1 support starting from SIENNA CICHLID asic
Signed-off-by: NArunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Starting from SIENNA CICHLID asic supports two gfx pipes, enabling
two graphics queues, 1 on each pipe, pipe0 queue0 would be the normal
piority queue and pipe1 queue0 would be the high priority queue

Only one queue per pipe is visble to SPI, SPI looks at the priority
value assigned to CP_GFX_HQD_QUEUE_PRIORITY from each of the queue's
HQD/MQD.

Create contexts applying AMDGPU_CTX_PRIORITY_HIGH which submits job
to the high priority queue on GFX pipe1. There would be starvation
of LP workload if HP workload is always available.

v2:
  - remove unnecessary check(Nirmoy)
  - make pipe1 hardware support a separate patch(Nirmoy)
  - remove duplicate code(Shashank)
  - add CSA support for second gfx pipe(Alex)

v3(Christian):
  - fix incorrect indentation
  - merge COMPUTE and GFX switch cases as both calls the same function.

v4:
  - rebase w/ latest code base
Signed-off-by: NArunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Spelling mistakes (triple letters) in comments.
Detected with the help of Coccinelle.
Signed-off-by: NJulia Lawall <Julia.Lawall@inria.fr>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Make sure the queue is not longer active before programming
the kiq EOP registers.
Signed-off-by: NHaohui Mai <ricetons@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Remove the accidental shifts on the values of RPTR_BLOCK_SIZE
in gfx_v8-v11. The bug essentially always programs the
corresponding fields to zero instead of the correct value.
The hardware clamps the min value to 5 so this resulted in a
value of 5 being programmed.
Signed-off-by: NHaohui Mai <ricetons@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Clean up redundant, copy-paste code blocks during the initialization of
the doorbells in mqd_init().
Signed-off-by: NHaohui Mai <ricetons@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This fixes a kernel oops when MES is not enabled.
Reported-by: NKenny Ho <Kenny.Ho@amd.com>
Suggested-by: NJack Xiao <Jack.Xiao@amd.com>
Reviewed-by: NAlex Deucher <Alexander.Deucher@amd.com>
Signed-off-by: NLuben Tuikov <luben.tuikov@amd.com>
Fixes: 18ee4ce6 ("drm/amdgpu: add mes unmap legacy queue routine")
Fixes: 3d879e81 ("drm/amdgpu: add init support for GFX11 (v2)")
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It's over a decade ago that this was actually used for more than ring and
IB tests. Just use the static register directly where needed and nuke the
now useless infrastructure.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NLang Yu <Lang.Yu@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Simplify the return expression.
Reported-by: NZeal Robot <zealci@zte.com.cn>
Signed-off-by: NMinghao Chi <chi.minghao@zte.com.cn>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This stops clang complaining:

drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c:3846:6: warning: variable 'index' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
        if (ring->is_mes_queue) {
            ^~~~~~~~~~~~~~~~~~
drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c:3903:30: note: uninitialized use occurs here
        amdgpu_device_wb_free(adev, index);
                                    ^~~~~
drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c:3846:2: note: remove the 'if' if its condition is always false
        if (ring->is_mes_queue) {
        ^~~~~~~~~~~~~~~~~~~~~~~~~
drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c:3839:16: note: initialize the variable 'index' to silence this warning
        unsigned index;
                      ^
                       = 0
Signed-off-by: NMike Lothian <mike@fireburn.co.uk>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable KIQ to map MES ring:
1). add MES queue mapping support in MAP_QUEUES packet.
2). use correct MQD settings for MES queue.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For mes kiq has been taken over by mes sched, drv can't directly
use mes kiq to unmap queues. drv has to use mes sched api to
unmap legacy queue.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable kiq support on gfx10.3, enable mes kiq (n-1)
test on sienna cichlid, so that mes kiq can be tested on
sienna cichlid. The patch can be dropped once mes kiq
is functional.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add mes support for gfx ib test.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

From IH ring buffer, look up the coresponding kernel queue and process.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For MES queue VM flush, use INVALIDATE_TLBS to invalidate TLBs.
This packet can let CP firmware to determine the current vmid
and inv eng to invalidate.

v2: unify invalidate_tlbs functions

Cc: Le Ma <le.ma@amd.com>
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For MES manages vmid assignment, let vmid inherit from mqd instead of
ib packet setting.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Associate mes queue id with fence, so that EOP trap handler can look up
which queue has issued the fence.

v2: move mes queue flag to amdgpu_mes_ctx.h
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

As MES requires per context preemption, use per context CSA address
for DE metadata to correctly enable context MCBP preemption.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

As MES requires per context preemption, use per context CSA address
for CE metadata to correctly enable context MCBP preemption.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Refine the existing gfx/compute mqd functions, and add them
to engine mqd layer.

v2: rebase fix.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Use ring structure to access the cpu/gpu address of rptr/wptr.

v2: merge gfx10/sdma5/sdma5.2 patches
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The gfx_v10_0_ring_test_ib() function uses 20 bytes instead of 16
bytes during the test. The patch sets the size of the allocation to be
4-byte larger to match the actual usage.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NHaohui Mai <ricetons@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of the 'amdgpu_ring_priority_level' type,
the 'amdgpu_gfx_pipe_priority' type was used,
which is an error when setting ring priority.
This is a minor error, but may cause problems in the future.

Instead of AMDGPU_RING_PRIO_2 = 2, we can use AMDGPU_RING_PRIO_MAX = 3,
but AMDGPU_RING_PRIO_2 = 2 is used for compatibility with
AMDGPU_GFX_PIPE_PRIO_HIGH = 2, and not change the behavior of the
code.
Signed-off-by: NGrigory Vasilyev <h0tc0d3@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

With this, we can support more CG flags.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

gfx10.3.3/gfx10.3.6/gfx10.3.7 shall use 0x1580 address for GCR_GENERAL_CNTL
Acked-by: NPrike Liang <Prike.Liang@amd.com>
Acked-by: NYifan Zhang <yifan1.zhang@amd.com>
Reviewed-by: NAaron Liu <aaron.liu@amd.com>
Signed-off-by: NRuili Ji <ruiliji2@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

gfx10.3.3/gfx10.3.6/gfx10.3.7 shall use 0x1580 address for GCR_GENERAL_CNTL
Acked-by: NPrike Liang <Prike.Liang@amd.com>
Acked-by: NYifan Zhang <yifan1.zhang@amd.com>
Reviewed-by: NAaron Liu <aaron.liu@amd.com>
Signed-off-by: NRuili Ji <ruiliji2@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable gfx1037 clock counter retrieval function for KFDPerfCountersTest.ClockCountersBasicTest.
Signed-off-by: NPrike Liang <Prike.Liang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Only GFX ring, SDMA ring and VCN decode ring support secure submission
at the moment.
Suggested-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NLang Yu <Lang.Yu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Leftover from bring up.  Not used anymore.
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable gfxoff routine for GC 10.3.7.
Signed-off-by: NPrike Liang <Prike.Liang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable gfx power gating for GC 10.3.7.
Signed-off-by: NPrike Liang <Prike.Liang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable gfx cg gate/ungate control for GC 10.3.7.
Signed-off-by: NPrike Liang <Prike.Liang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

this patch adds gc 10.3.6 support.
Signed-off-by: NYifan Zhang <yifan1.zhang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Needed to properly initialize GC 10.3.7.
Signed-off-by: NPrike Liang <Prike.Liang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Since this is an existing asic, adjust the code to follow
the same logic as previously so the driver state is consistent.

No functional change intended.
Acked-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add basic support for GC 10.1.4,
it uses same IP blocks with GC 10.1.3
Signed-off-by: NLang Yu <Lang.Yu@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

"adev->gfx.rlc.rlcg_reg_access_supported = true;"
the above varible were set too late during driver initialization.
it will cause the driver to fail to write/read register during GMC hw init
in sriov mode.

move gfx_xxx_init_rlcg_reg_access_ctrl() function to gfx early init stage
to avoid this issue.

Fixes: 5d447e29 ("drm/amdgpu: add helper for rlcg indirect reg access")
Signed-off-by: NYang Wang <KevinYang.Wang@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>