Currently, we have a limitted GTT memory size and need a bounce buffer
when doing buffer migration between VRAM and SYSTEM domain.

The problem is under GTT memory pressure we can't do buffer migration
between VRAM and SYSTEM domain. But in some cases we really need that.
Eespecially when validating a VRAM backing store BO which resides in
SYSTEM domain.

v2: still account temporary GTT allocations
v3 (chk): revert to the simpler change for now
Signed-off-by: NLang Yu <Lang.Yu@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Acked-by: NNirmoy Das <nirmoy.das@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210622162339.761651-2-andrey.grodzovsky@amd.com

Drop the workaround and instead implement a better solution.

Basically we are now chaining all submissions using a dma_fence_chain
container and adding them as exclusive fence to the dma_resv object.

This way other drivers can still sync to the single exclusive fence
while amdgpu only sync to fences from different processes.

v3: add the shared fence first before the exclusive one
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210614174536.5188-2-christian.koenig@amd.com

Unwrap the explicit fence if it is a dma_fence_chain and
sync to the first fence not matching the owner rules.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210614174536.5188-1-christian.koenig@amd.com

There is a reversed if statement in amdgpu_preempt_mgr_new() so it
always returns -ENOMEM.

Fixes: 09b020bb ("Merge tag 'drm-misc-next-2021-06-09' of git://anongit.freedesktop.org/drm/drm-misc into drm-next")
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/YMxbQXg/Wqm0ACxt@mwandaSigned-off-by: NChristian König <christian.koenig@amd.com>

Add a common allocation helper. Cleaning up the mix of kzalloc/kmalloc
and some unused code in the selftest.

v2: polish kernel doc a bit
v3: polish kernel doc even a bit more
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210611120301.10595-3-christian.koenig@amd.com

This fixes 32-bit arm build due to lack of 64-bit divides.

Fixes: cb1c8146 ("drm/ttm: flip the switch for driver allocated resources v2")
Link: https://patchwork.freedesktop.org/patch/438442/Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Drop disabling of gfxoff during VCN use.  This allows gfxoff
to kick in and potentially save power if the user is not using
gfx for color space conversion or scaling.

VCN1.0 had a bug which prevented it from working properly with
gfxoff, so we disabled it while using VCN.  That said, most apps
today use gfx for scaling and color space conversion rather than
overlay planes so it was generally in use anyway. This was fixed
on VCN2+, but since we mostly use gfx for color space conversion
and scaling and rapidly powering up/down gfx can negate the
advantages of gfxoff, we left gfxoff disabled. As more
applications use overlay planes for color space conversion
and scaling, this starts to be a win, so go ahead and leave
gfxoff enabled.

Note that VCN1.0 uses vcn_v1_0_idle_work_handler() and
vcn_v1_0_ring_begin_use() so they are not affected by this
patch.
Reviewed-by: NJames Zhu <James.Zhu@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBoyuan Zhang <Boyuan.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is similar to IH_RB_CNTL programming in
navi10_ih_toggle_ring_interrupts
Signed-off-by: NRohit Khaire <rohit.khaire@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHorace Chen <horace.chen@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

psp added new feature to check fw buffer address for sriov vf. the
address range must be in vf fb.
Signed-off-by: NZhigang Luo <zhigang.luo@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-By : Shaoyun.liu <shaoyunl@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

need to load xgmi ta for aldebaran sriov vf.
Signed-off-by: NZhigang Luo <zhigang.luo@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-By : Shaoyun.liu <shaoyunl@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

host driver programmed mmhub system aperture and fb location for vf, no
need to program in guest side.
Signed-off-by: NZhigang Luo <zhigang.luo@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-By : Shaoyun.liu <shaoyunl@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

host driver programmed the gfxhub fb location for vf, no need to
program in guest side.
Signed-off-by: NZhigang Luo <zhigang.luo@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-By : Shaoyun.liu <shaoyunl@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

host driver programmed fb location registers for vf, no need to
check anymore.
Signed-off-by: NZhigang Luo <zhigang.luo@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-By : Shaoyun.liu <shaoyunl@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Create shadow BOs only for no-compute VM context and only for dGPU.
The existing if-condition would create shadow bo for compute context
on dGPU which not what we wanted.
Signed-off-by: NNirmoy Das <nirmoy.das@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

s/than/then/
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add the parameter table_freed description on function description.
Signed-off-by: NEric Huang <jinhuieric.huang@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For GTT allocations with a GART address the res contains the VMID0
addresses and can't be used for VM handling.

So ignore the res when the pages array is given or we fill the page
tables with nonsense.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NNirmoy Das <nirmoy.das@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The NV12 and VEGA10 share the same interface W/RREG32_SOC15*,
the callback functions in these macros may not be defined,
so NULL pointer must be checked but not in
macro __WREG32_SOC15_RLC__, fixing the lock of NULL pointer check.
Signed-off-by: NPeng Ju Zhou <PengJu.Zhou@amd.com>
Reviewed-by: NEmily Deng <Emily.Deng@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Disable support on APU
Reviewed-by: NChangfeng <Changfeng.Zhu@amd.com>
Signed-off-by: NJohn Clements <john.clements@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

In SRIOV environment, KMD should access SDMA registers
through RLCG if GC indirect access flag enabled.

Using _SOC15 read/write macros ensures that they go
through RLC when the flag is enabled.
Signed-off-by: NRohit Khaire <rohit.khaire@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We are calling the eviction_valuable driver callback at eviction time to
determine whether we actually can evict a buffer object.
The upcoming i915 TTM backend needs the same functionality for swapout,
and that might actually be beneficial to other drivers as well.

Add an eviction_valuable call also in the swapout path. Try to keep the
current behaviour for all drivers by returning true if the buffer object
is already in the TTM_PL_SYSTEM placement. We change behaviour for the
case where a buffer object is in a TT backed placement when swapped out,
in which case the drivers normal eviction_valuable path is run.
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Link: https://lore.kernel.org/r/20210602083818.241793-8-thomas.hellstrom@linux.intel.com
Link: https://patchwork.freedesktop.org/patch/msgid/20210602083818.241793-8-thomas.hellstrom@linux.intel.com

The functions can be called both in _rcu context as well
as while holding the lock.

v2: add some kerneldoc as suggested by Daniel
v3: fix indentation
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJason Ekstrand <jason@jlekstrand.net>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210602111714.212426-7-christian.koenig@amd.com

When the comment needs to state explicitly that this is doesn't get a reference
to the object then the function is named rather badly.

Rename the function and use it in even more places.

v2: use dma_resv_shared_list as new name
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210602111714.212426-5-christian.koenig@amd.com

When the comment needs to state explicitly that this
doesn't get a reference to the object then the function
is named rather badly.

Rename the function and use rcu_dereference_check(), this
way it can be used from both rcu as well as lock protected
critical sections.

v2: improve kerneldoc as suggested by Daniel
v3: use dma_resv_excl_fence as function name
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NJason Ekstrand <jason@jlekstrand.net>
Link: https://patchwork.freedesktop.org/patch/msgid/20210602111714.212426-4-christian.koenig@amd.com

To enable output on real display instead of virtual.
Acked-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NNicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add video_codecs query support for yellow carp.
Acked-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This patch adds mode2 reset support for yellow carp.
Signed-off-by: NAaron Liu <aaron.liu@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Skip disabled sa to correct the cu_info and active_rbs for yellow carp.
Signed-off-by: NXiaomeng Hou <Xiaomeng.Hou@amd.com>
Suggested-by: NAaron Liu <aaron.liu@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Register callback in gfxhub functions to program the bypass groups in
gc_utcl2 corresponding to harvested SA.

v2: update comments (Alex)
Signed-off-by: NXiaomeng Hou <Xiaomeng.Hou@amd.com>
Signed-off-by: NAaron Liu <aaron.liu@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add TA firmware to module firmware list for yellow carp and call
psp_init_ta_microcode to parse the TA firmware for HDCP support.

Cc: Aaron Liu <aaron.liu@amd.com>
Acked-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NNicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Allows software to query HW counters to timestamp submissions.
This patch can address KFDPerfCountersTest.
Signed-off-by: NAaron Liu <aaron.liu@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: Nchen gong <curry.gong@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

RLC_PG_DELAY_3 is to make RLC in safe mode to
prevent any misalignment or conflict in middle of any power
feature entry/exit sequence when CGPG feature is enabled.
Signed-off-by: NAaron Liu <aaron.liu@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable VCN 3.0 PG and CG for Yellow Carp by setting up flags.
Acked-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAaron Liu <aaron.liu@amd.com>
Reviewed-by: NJames Zhu <James.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable vcn dpg mode on yellow carp.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NLeo Liu <leo.liu@amd.com>
Reviewed-by: NAaron Liu <aaron.liu@amd.com>
Acked-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable vcn/jpeg IP on yellow carp.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NLeo Liu <leo.liu@amd.com>
Reviewed-by: NAaron Liu <aaron.liu@amd.com>
Acked-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add vcn firmware support for yellow carp
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NLeo Liu <leo.liu@amd.com>
Reviewed-by: NAaron Liu <aaron.liu@amd.com>
Acked-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Register CC_UVD_HARVESTING is obsolete on CHIP_YELLOW_CARP.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NLeo Liu <leo.liu@amd.com>
Reviewed-by: NAaron Liu <aaron.liu@amd.com>
Acked-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

IH CG need to be enabled by driver.
Signed-off-by: NAaron Liu <aaron.liu@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

ATHUB MGCG/MGLS is enabled by default.
Adding ATHUB MGCG/MGLS flag to ensure athub mgcg/ls enabled.
Signed-off-by: NAaron Liu <aaron.liu@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

HDP MGCG is enabled by default.
Adding AMD_CG_SUPPORT_HDP_MGCG to ensure hdp mgcg enabled.
HDP MGLS need to be enabled by driver.
Signed-off-by: NAaron Liu <aaron.liu@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>