As part of the ib padding process, accessing the RLC_SPM_* register may
trigger gfx hang. Since gfxoff may be already kicked during the whole period.
To address that, we manually toggle gfx on/off around the RLC_SPM_*
register access.

This can resolve the gfx hang issue observed on running Talos with RDP launched
in parallel.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NGuchun Chen <guchun.chen@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The error count reset for xgmi3x16 pcs is missed.
Signed-off-by: NTao Zhou <tao.zhou1@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Previously there was a check based on chip # for chips that aligned to
>=CHIP_NAVI10 to have RLC stopped as part of DPMS check.  This was because
of gfxclk being controlled by RLC in the newer designs.

As part of IP version checking though, this got changed to match IP
version for SMU.  Because Renoir designs also include smu11 that meant
that even GFX9 started to stop RLC earlier.

Adjust to match GFX IP version instead of SMU IP version to restore the
previous behavior.

Fixes: a8967967 ("drm/amdgpu/amdgpu_smu: convert to IP version checking")
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
csb bo is not unpinned in gfx 9. It will lead to pin_count leak on
driver unload.

[How]
Call bo_free_kernel corresponding to bo_create_kernel in
gfx_rlc_init_csb. This will also unify the code path with other gfx
versions.
Signed-off-by: NYuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
For Vega10, disabling gart of gfxhub could mess up KIQ and PSP
under sriov mode, and lead to DMAR on host side.

[How]
Skip writing GMC registers under sriov.
Signed-off-by: NYuBiao Wang <YuBiao.Wang@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

sysfs_emit and sysfs_emit_at requrie a page boundary
aligned buf address. Make them happy!

v2: fix sysfs_emit -> sysfs_emit_at missed conversions

Cc: Lang Yu <lang.yu@amd.com>
Cc: Darren Powell <darren.powell@amd.com>
Fixes: 6db0c87a ("amdgpu/pm: Replace hwmgr smu usage of sprintf with sysfs_emit")
Bug: https://gitlab.freedesktop.org/drm/amd/-/issues/1774Reviewed-by: NLang Yu <lang.yu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

BOs need to be reserved before they are added or removed, so ensure that
they are reserved during kfd_mem_attach and kfd_mem_detach
Signed-off-by: NKent Russell <kent.russell@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]:
When we call hmm_range_fault to map memory after a migration, we don't
expect memory to be migrated again as a result of hmm_range_fault. The
driver ensures that all memory is in GPU-accessible locations so that
no migration should be needed. However, there is one corner case where
hmm_range_fault can unexpectedly cause a migration from DEVICE_PRIVATE
back to system memory due to a write-fault when a system memory page in
the same range was mapped read-only (e.g. COW). Ranges with individual
pages in different locations are usually the result of failed page
migrations (e.g. page lock contention). The unexpected migration back
to system memory causes a deadlock from recursive locking in our
driver.

[How]:
Creating a task reference new member under svm_range_list struct.
Setting this with "current" reference, right before the hmm_range_fault
is called. This member is checked against "current" reference at
svm_migrate_to_ram callback function. If equal, the migration will be
ignored.
Signed-off-by: NAlex Sierra <alex.sierra@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

There is no reason to allow for partial buffers from userspace in our
debugfs. In this particular case callers will zero out the wr_buf but if
callers in the future don't do that we might be looking at corrupt data.

Linus puts it better than I can in
https://lkml.org/lkml/2021/10/26/993Reviewed-by: NRodrigo Siqueira <Rodrigo.Siqueira@amd.com>
Signed-off-by: NHarry Wentland <harry.wentland@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NAric Cyr <aric.cyr@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NAnthony Koo <Anthony.Koo@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why & How]
In order to have dc_enable_dmub_notifications() more precise, add
one more condition to check if dc->debug.dpia_debug.bits.disable_dpia
is false.
Signed-off-by: NMeenakshikumar Somasundaram <meenakshikumar.somasundaram@amd.com>
Signed-off-by: NWayne Lin <Wayne.Lin@amd.com>
Signed-off-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
During Z10, root clock gating and memory low power registers needs to
to be restored if optimization is enabled in driver.

[How]
Added new DMUB boot option for root clock gating and memory low power.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NJake Wang <haonan.wang2@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why & How]
The MPC memory clocks should be powered down when not in use.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NJake Wang <haonan.wang2@amd.com>
Reviewed-by: NEric Yang <eric.yang2@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
HPO is only used for DP2.0. HPO HW control should be
disable when not being used to save power.

[How]
Shutdown HPO HW control during init hw.
Shutdown HPO HW control during stream disable.
Enable HPO HW control during stream enable if DP2.0.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NJake Wang <haonan.wang2@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why&how]
write LINK_SQUARE_PATTERN_num + 1 for square pulse pattern.
Specs requirement to write this register prior to write LINK_QUAL_LANEX_SET.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Reviewed-by: NGeorge Shen <george.shen@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
State can be cleared without removing individual streams (by
calling dc_remove_stream_from_ctx()). This can leave the
encoder assignment module in an incoherent state and cause
future assignments to be incorrect.

[How]
Clear encoder assignments when committing 0 streams or
re-initializing hardware.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NJimmy Kizito <Jimmy.Kizito@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
In scenario when 1 display connected with pipe split (2 pipes in use)
and 3 new displays simultaneously hotplugged via MST hub (4 pipes in use),
mpcc may get reprogram to other vtg, remaining busy.
In this case waiting for mpcc idle timeouts with error like this:
[drm] REG_WAIT timeout 1us * 100000 tries - mpc2_assert_idle_mpcc
RIP: 0010:mpc2_assert_mpcc_idle_before_connect
Call Trace:
dcn20_update_mpcc
dcn20_program_front_end_for_ctx
dc_commit_state
amdgpu_dm_atomic_commit_tail
...

[How]
Add pipe split change condition to disable dangling plane.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NRoman Li <Roman.Li@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Reviewed-by: NDmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
1. YCbCr 4:2:2 8bpc/10bpc modes are blocked for HDMI by policy
2. A YCbCr 4:2:0 calculation error blocked some 4:2:0 timing modes

[How]
YCbCr 4:2:2 8bpc/10bpc modes are allowed for HDMI
Fix YCbCr 4:2:0 calculation error
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NBing Guo <Bing.Guo@amd.com>
Reviewed-by: NChris Park <chris.park@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
MST monitor sends link loss short pulse continuous but sink is
occupy by HDMI input to lead link training fail.

[How]
disable link once retraining fail.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NYu-ting Shen <yu-tshen@amd.com>
Reviewed-by: NWenjing Liu <Wenjing.Liu@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
It was found that the system would hang on a dummy pstate when playing
4k60 videos on a 1080p 390Hz monitor.

[HOW]
Properly select the dummy_pstate_latency_ms when firmware assisted
memory clock switching is enabled instead of assuming that the highest
latency would work for every monitor timing.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NFelipe Clark <felclark@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Fix spacing issue for the format string.

Addresses-Coverity-ID: 1446765: ("Invalid printf format string")
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NAnson Jacob <Anson.Jacob@amd.com>
Reviewed-by: NRodrigo Siqueira <Rodrigo.Siqueira@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
If timing and bpp of displays on mst hub are not changed,
pbn, slot_num for displays should not be changed. Linux
user mode may initiate atomic_check with different display
configuration after set mode finished. This will call to
amdgpu_dm to re-compute payload, slot_num of displays and
saved to dm_connect_state. stream->timing.flags.dsc, pbn,
slot_num are updated to values which may be different from
that were used for set mode. when dsc hub with 3 4k@60hz dp
connected, 3 dsc engines are enabled. timing.flags.dsc = 1.
timing.flags.dsc are changed to 0 due to atomic check. when
dsc hub is unplugged, amdgpu driver check timing.flags.dsc
for last mode set and find out flags.dsc = 0, then does not
disable dsc.

[HOW]
check status of  displays on dsc mst hubs. re-compute pbn,
slot_num, timing.flags.dsc only if there is mode, connect
or enable/disable change.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NHersen Wu <hersenwu@amd.com>
Reviewed-by: NMikita Lipski <Mikita.Lipski@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Remove duplicated kfd_resume_iommu which already runs
in mdgpu_amdkfd_device_init.
Tested-By: NKen Moffat <zarniwhoop@ntlworld.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For yellow carp, the desired CGPG hysteresis value is 0x4E20.
Signed-off-by: NAaron Liu <aaron.liu@amd.com>
Acked-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

commit 652de07a ("drm/amd/display: Fully switch to dmub for all dcn21
asics") switched over to using dmub on Renoir to fix Gitlab 1735, but this
implied a new dependency on newer firmware which might not be met on older
kernel versions.

Since sw_init runs before hw_init, there is an opportunity to determine
whether or not the firmware version is new to adjust the behavior.

Cc: Roman.Li@amd.com
BugLink: https://gitlab.freedesktop.org/drm/amd/-/issues/1772
BugLink: https://gitlab.freedesktop.org/drm/amd/-/issues/1735
Fixes: 652de07a ("drm/amd/display: Fully switch to dmub for all dcn21 asics")
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NRoman Li <Roman.Li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For ASICs not supporting power profile mode, don't show the attribute.
Verify that the function has been implemented by the subsystem.
Suggested-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This better aligns that the caller can make a mistake with the buffer
and -EINVAL should be returned, but if the hardware doesn't support
the feature it should be -EOPNOTSUPP.
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Prevent possible issues from set and get being called simultaneously.
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This was added by commit bd8dcea9 ("drm/amd/pm: add callbacks to
read/write sysfs file pp_power_profile_mode") but the feature was
deprecated from PMFW.  Remove it from the driver.
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Previously Renoir compiler gfx target version was forced to Raven.
Update driver side for completeness.
Signed-off-by: NGraham Sider <Graham.Sider@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is more useful when talking to the SMU team to have the information
in this format, save one less step to manually do it.
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

If some pages fail to migrate to system memory, don't update
prange->actual_loc = 0. This prevents endless CPU page faults after
partial migration failures due to contested page locks.

Migration to RAM must be complete during migrations from VRAM to VRAM and
during evictions. Implement retry and fail if the migration to RAM fails.
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NPhilip Yang <Philip.Yang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Stack and heap pages tend to be shared by many small allocations.
Concurrent access by CPU and GPU is therefore likely, which can lead to
thrashing. Avoid this by setting the preferred location to system memory.
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NPhilip Yang <Philip.Yang@amd.com>
Reviewed-by: NPhilip Yang <Philip.Yang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The preferred location should be used as the migration destination
whenever it is accessible by the faulting GPU. System memory is always
accessible. Peer memory is accessible if it's in the same XGMI hive.
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NPhilip Yang <Philip.Yang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Aldebaran has different register mask definitions for
regiter MC_VM_XGMI_LFB_CNTL. Use the correct masks
to interpret fields of this register.
Signed-off-by: NOak Zeng <Oak.Zeng@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
In advance tdr mode, the real bad job will be resubmitted twice, while
in drm_sched_resubmit_jobs_ext, there's a dma_fence_put, so the bad job
is put one more time than other jobs.

[How]
Adding dma_fence_get before resbumit job in
amdgpu_device_recheck_guilty_jobs and put the fence for normal jobs
Signed-off-by: NJingwen Chen <Jingwen.Chen2@amd.com>
Reviewed-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Merge tag 'amd-drm-next-5.16-2021-10-29' of https://gitlab.freedesktop.org/agd5f/linux into drm-next

amd-drm-next-5.16-2021-10-29:

amdgpu:
- RAS fixes
- Fix a potential memory leak in device tear down
- Add a stutter mode quirk
- Misc display fixes
- Further display FP refactoring
- Display USB4 fixes
- Display DP2.0 fixes
- DCN 3.1 fixes
- Display 8 ch audio fix
- Fix DMA mask regression for SI parts
- Aldebaran fixes

amdkfd:
- userptr fix
- BO lifetime fix
- Misc code cleanup

UAPI:
- Minor header cleanup (no functional change)
Signed-off-by: NDave Airlie <airlied@redhat.com>
From: Alex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20211029184338.4863-1-alexander.deucher@amd.com

The DMA mask on SI parts is 40 bits not 44.  Copy
paste typo.

Fixes: 244511f3 ("drm/amdgpu: simplify and cleanup setting the dma mask")
Bug: https://gitlab.freedesktop.org/drm/amd/-/issues/1762Acked-by: NChristian König <christian.koenig@amd.com>
Tested-by: NPaul Menzel <pmenzel@molgen.mpg.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
- DPIA MST slot registers are not programmed during payload
allocation and hence MST does not work with DPIA.
- HPD RX interrupts are not handled for DPIA.

[How]
- Added inbox command to program the MST slots whenever
  payload allocation happens for DPIA links.
- Added support for handling HPD RX interrupts
Signed-off-by: NMeenakshikumar Somasundaram <meenakshikumar.somasundaram@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NNicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>