For MES queue VM flush, use INVALIDATE_TLBS to invalidate TLBs.
This packet can let CP firmware to determine the current vmid
and inv eng to invalidate.

v2: unify invalidate_tlbs functions

Cc: Le Ma <le.ma@amd.com>
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For MES manages vmid assignment, let vmid inherit from mqd instead of
ib packet setting.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Associate mes queue id with fence, so that EOP trap handler can look up
which queue has issued the fence.

v2: move mes queue flag to amdgpu_mes_ctx.h
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

As MES requires per context preemption, use per context CSA address
for DE metadata to correctly enable context MCBP preemption.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

As MES requires per context preemption, use per context CSA address
for CE metadata to correctly enable context MCBP preemption.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Refine the existing gfx/compute mqd functions, and add them
to engine mqd layer.

v2: rebase fix.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

assign the cpu/gpu address of fence for the normal or mes ring
from ring structure.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Iniailize/finalize the ring for mes queue which submits the command
stream to the mes-managed hardware queue.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Use ring structure to access the cpu/gpu address of rptr/wptr.

v2: merge gfx10/sdma5/sdma5.2 patches
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Define ring structure to access the cpu/gpu address of rptr/wptr/fence
instead of dynamic calculation.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add mes context data structure in amdgpu_ring.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The per-context meta data is a per-context data structure associated
with a mes-managed hardware ring, which includes MCBP CSA, ring buffer
and etc.

v2: fix typo
v3: a. use structure instead of typedef
    b. move amdgpu_mes_ctx_get_offs_* to amdgpu_ring.h
    c. use __aligned to make alignement
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add the helper function to initialize mqd from ring configuration.

v2: use if/else pair instead of ?/: pair
v3: use simpler way to judge hqd_active
v4: fix parameters to amdgpu_gfx_is_high_priority_compute_queue
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Define MQD abstract layer for hw ip, for the passing
mqd configuration not only from ring but more sources,
like user queue.
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add IMU firmware structure.
Signed-off-by: NLikun Gao <Likun.Gao@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add RLC autoload TOC header file for soc21 ASIC.

v2: squash in updates
Signed-off-by: NLikun Gao <Likun.Gao@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add the CG flag for Fine Grained Clock Gating.
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add support for rlc v2_3 to support RLCV and RLCP fw load.
Signed-off-by: NLikun Gao <Likun.Gao@amd.com>
Reviewed-by: NKenneth Feng <kenneth.feng@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We need define new firmware header to support
CP RS64 fw.
Signed-off-by: NWenhui Sheng <Wenhui.Sheng@amd.com>
Signed-off-by: NLikun Gao <Likun.Gao@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NWenhui Sheng <Wenhui.Sheng@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

memory queue descriptors for gfx11.
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NLikun Gao <Likun.Gao@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add gfx11 clearstate register arrays
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

add soc21 enum definitions (Hawking)
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NLikun Gao <Likun.Gao@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NLikun Gao <Likun.Gao@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Set GC family for GC 11.0 IPs.
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NLikun Gao <Likun.Gao@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add GC v11_0_0 family id
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NLikun Gao <Likun.Gao@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When psp_hw_init failed, it will set the load_type to AMDGPU_FW_LOAD_DIRECT.
During amdgpu_device_ip_fini, amdgpu_ucode_free_bo checks that load_type is
AMDGPU_FW_LOAD_DIRECT and skips deallocating fw_buf causing memory leak.
Remove load_type check in amdgpu_ucode_free_bo.
Signed-off-by: NAlice Wong <shiwei.wong@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Recently we got a hard hang during the boot on DCN 3.0.1,
which caused the below null pointer exception:

[ +0.000426] BUG: kernel NULL pointer dereference, address: 0000000000000000
[ +0.000003] #PF: supervisor read access in kernel mode
[ +0.000003] #PF: error_code(0x0000) - not-present page
[ +0.000003] PGD 0 P4D 0
[ +0.000004] Oops: 0000 [#1] PREEMPT SMP NOPTI
[ +0.000005] CPU: 6 PID: 874 Comm: Xorg Not tainted 5.16.0.asdn-apr28+ #15
[ +0.000004] Hardware name: AMD Chachani-VN/Chachani-VN, BIOS WCH2303N 03/03/2022
[ +0.000003] RIP: 0010:resource_map_pool_resources+0x431/0xa70 [amdgpu]
[ +0.000356] Code: c1 4d 89 c8 49 c1 e0 07 4d 01 c8 49 c1 e0 04 4d 01 f0 49 83 b8 f0 01 00 00 00 0f 85 16 02 00 00 49 8b b8 e0 02 00 00 89 45 c0 <48> 8b 17 4c 8b 92 a0 01 00 00 4d 85 d2 74 24 4c 89 4d 88 48 8d 4d
[ +0.000003] RSP: 0018:ffffa92a4142f718 EFLAGS: 00010246
[ +0.000003] RAX: 0000000000000000 RBX: ffff9a0b86d93000 RCX: 0000000000000000
[ +0.000002] RDX: 0000000000000000 RSI: 000000000000554b RDI: 0000000000000000
[ +0.000002] RBP: ffffa92a4142f798 R08: ffff9a0bdb3c0000  0000000000000000
[ +0.000002] R10: 0000000000000000 R11: 000000000000f000 R12: 0000000000000000
[ +0.000001] R13: ffff9a0b88360000 R14: ffff9a0bdb3c0000 R15: ffff9a0b86273000
[ +0.000003] FS: 00007f4b5641ca40(0000) GS:ffff9a0cb7f80000(0000) knlGS:0000000000000000
[ +0.000002] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ +0.000002] CR2: 0000000000000000 CR3: 0000000102cb2000 CR4: 00000000003506e0
[ +0.000003] Call Trace:
[ +0.000002] <TASK>
[ +0.000004] ? kvmalloc_node+0x5c/0x90
[ +0.000009] dcn20_add_stream_to_ctx+0x1c/0x90 [amdgpu]
[ +0.000330] dcn30_add_stream_to_ctx+0xe/0x10 [amdgpu]
[ +0.000313] dc_add_stream_to_ctx+0x67/0x80 [amdgpu]
[ +0.000300] dm_update_crtc_state+0x4dd/0x6e0 [amdgpu]
[ +0.000320] amdgpu_dm_atomic_check+0x63b/0x1270 [amdgpu]
[ +0.000311] ? __drm_mode_object_add+0x90/0xc0 [drm]
[ +0.000043] ? preempt_count_add+0x74/0xc0
[ +0.000005] ? _raw_spin_lock_irqsave+0x2a/0x60
[ +0.000006] ? _raw_spin_unlock_irqrestore+0x29/0x3d
[ +0.000003] ? drm_connector_list_iter_next+0x8e/0xb0 [drm]
[ +0.000038] drm_atomic_check_only+0x5dd/0xa20 [drm]
[ +0.000044] drm_atomic_commit+0x18/0x60 [drm]
[ +0.000046] drm_client_modeset_commit_atomic+0x1e5/0x220 [drm]
[ +0.000051] drm_client_modeset_commit_locked+0x57/0x160 [drm]
[ +0.000038] __drm_fb_helper_restore_fbdev_mode_unlocked+0x60/0xd0 [drm_kms_helper]
[ +0.000027] drm_fb_helper_set_par+0x40/0x50 [drm_kms_helper]
[ +0.000022] fb_set_var+0x1c8/0x3d0
[ +0.000007] ? __ext4_mark_inode_dirty+0x83/0x210
[ +0.000006] ? __ext4_journal_stop+0x3c/0xb0
[ +0.000008] fbcon_blank+0x228/0x290
[ +0.000007] do_unblank_screen+0xae/0x150
[ +0.000005] vt_ioctl+0xcf4/0x1360
[ +0.000005] ? get_max_files+0x20/0x20
[ +0.000005] ? get_max_files+0x20/0x20
[ +0.000004] ? debug_smp_processor_id+0x17/0x20
[ +0.000004] tty_ioctl+0x373/0x8a0
[ +0.000005] ? __fput+0x123/0x260
[ +0.000004] ? __fget_light+0xc5/0x100
[ +0.000005] __x64_sys_ioctl+0x91/0xc0
[ +0.000005] do_syscall_64+0x3b/0xc0
[ +0.000005] entry_SYSCALL_64_after_hwframe+0x44/0xae

This issue happens because "pipe_ctx->stream_res.tg"
needs to be initialized first before reading its members.
This commit fixes this issue by properly initializing
the pointer before accessing the target data.

Fixes: 663d2dae ("drm/amd/display: Add odm seamless boot support")
Cc: Agustin Gutierrez <agustin.gutierrez@amd.com>
Signed-off-by: NSung Joon Kim <Sungjoon.Kim@amd.com>
Reviewed-by: NAgustin Gutierrez <agustin.gutierrez@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable SMU on SMU IP version 13.0.0
Signed-off-by: NLikun Gao <Likun.Gao@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable ATHUB and MMHUB powergating.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Modify the common smu13 code and add a new smu
13.0.0 ppt file to handle the smu 13.0.0 specific
configuration.

v2: squash in typo fix in profile name
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add driver_if ppsmc and pptable header files
for smu v13_0_0
Signed-off-by: NLikun Gao <Likun.Gao@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We need to relay on this way to get the raw PPTable when
SCPM feature is enabled.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Correct the hw initialization sequence.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

With SCPM enabled, the pptable used will be signed. It cannot
be used directly by driver. To get the raw pptable, we need to
rely on the combo pptable(and its revelant SMU message).

Also, the pptable transferring(to SMU) will be performed by PSP.
Some SMU messages will be not available to driver any more.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

With SCPM enabled, pptable cannot be uploaded to SMU directly.
The transferring has to be via PSP.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Place pptable setup after smu_set_driver_table_location. As under SCPM
enabled scenario, the latter one is a prerequisite for the former one.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The bootup clocks can be retrieved from the smu_info table. That
is light-weight compared with existing way.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NKenneth Feng <kenneth.feng@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Firsrt of all, the operations involved is to interact with
VBIOS. They are fully supported at ->sw_init phase.

Secondly, the new mechanism to upload pptable to SMU is
introduced. With the new mechanism, the pptable transferring
has to be via PSP. That requires the pptable ucode(and necessary
bootupp values retrieving) must be ready before ->hw_init phase of PSP.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Fix NULL pointer issue on amdgpu_smu_stb_debug_fs_init if SMU block not
enabled.
Signed-off-by: NLikun Gao <Likun.Gao@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Make the pp_dpm_vclk/dclk sysfs interfaces visible for SMU 13.0.0.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>