The check for whether to drain retry faults must be under the mmap write
lock to serialize with munmap notifier callbacks.

We were also missing checks on child ranges. To fix that, simplify the
logic by using a flag rather than checking on each prange. That also
allows draining less freqeuntly when many ranges are unmapped at once.
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Tested-by: NPhilip Yang <Philip.Yang@amd.com>
Tested-by: NAlex Sierra <Alex.Sierra@amd.com>
Reviewed-by: NPhilip Yang <Philip.Yang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The low 16MB of virtual address space are currently reserved for kernel
mode allocations mapped into user virtual address space. This causes
conflicts with HMM/SVM mappings at low virtual addresses. We tried to
move those kernel mode allocations to the upper half of the 64-bit
virtual address space for GFX9, which is naturally reserved for kernel
use. However, TBA (trap handler code) has problems to access addresses
in the high virtual space. We have decided to set this to 8KB of the
lower address space as a temporary fix, while investigate TBA address
problem. It is very unlikely for user space to map memory at this low
region.
Signed-off-by: NAlex Sierra <alex.sierra@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

make action upon failure in "drm_atomic_add_affected_connectors()"
consistent with the rest of failures in amdgpu_dm_atomic_check().
Signed-off-by: NShirish S <shirish.s@amd.com>
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The KFD pre_reset should be called before reset been executed, it will
hold the lock to prevent other rocm process to sent the packlage to hiq
during host execute the real reset on the HW
Signed-off-by: Nshaoyunl <shaoyun.liu@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

There was a change(below) target for such issue:
d82e2c24 ("drm/amdgpu: Fix crash on device remove/driver unload")
But the fix for VI ASICs was missing there. This is a supplement for
that.

Fixes: d82e2c24 ("drm/amdgpu: Fix crash on device remove/driver unload")
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Properly handle SI DC support when CONFIG_DRM_AMD_DC_SI is not
set.

Fixes: f7f12b25 ("drm/amdgpu: default to true in amdgpu_device_asic_has_dc_support")
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

If a kfd_bo was shared (e.g. a dmabuf export), the original kfd_bo may be
freed when the amdgpu_bo still lives on. Free the kfd_bo struct in the
release_notify callback then the amdgpu_bo is freed.
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-By: NRamesh Errabolu <Ramesh.Errabolu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When kfd need to be reset, sent command to HWS might cause hang and get unnecessary timeout.
This change try not to touch HW in pre_reset and keep queues to be in the evicted state
when the reset is done, so they are not put back on the runlist. These queues will be destroied
on process termination.
Signed-off-by: Nshaoyunl <shaoyun.liu@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

As part of the ib padding process, accessing the RLC_SPM_* register may
trigger gfx hang. Since gfxoff may be already kicked during the whole period.
To address that, we manually toggle gfx on/off around the RLC_SPM_*
register access.

This can resolve the gfx hang issue observed on running Talos with RDP launched
in parallel.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NGuchun Chen <guchun.chen@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The error count reset for xgmi3x16 pcs is missed.
Signed-off-by: NTao Zhou <tao.zhou1@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Previously there was a check based on chip # for chips that aligned to
>=CHIP_NAVI10 to have RLC stopped as part of DPMS check.  This was because
of gfxclk being controlled by RLC in the newer designs.

As part of IP version checking though, this got changed to match IP
version for SMU.  Because Renoir designs also include smu11 that meant
that even GFX9 started to stop RLC earlier.

Adjust to match GFX IP version instead of SMU IP version to restore the
previous behavior.

Fixes: a8967967 ("drm/amdgpu/amdgpu_smu: convert to IP version checking")
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
csb bo is not unpinned in gfx 9. It will lead to pin_count leak on
driver unload.

[How]
Call bo_free_kernel corresponding to bo_create_kernel in
gfx_rlc_init_csb. This will also unify the code path with other gfx
versions.
Signed-off-by: NYuBiao Wang <YuBiao.Wang@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
For Vega10, disabling gart of gfxhub could mess up KIQ and PSP
under sriov mode, and lead to DMAR on host side.

[How]
Skip writing GMC registers under sriov.
Signed-off-by: NYuBiao Wang <YuBiao.Wang@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

sysfs_emit and sysfs_emit_at requrie a page boundary
aligned buf address. Make them happy!

v2: fix sysfs_emit -> sysfs_emit_at missed conversions

Cc: Lang Yu <lang.yu@amd.com>
Cc: Darren Powell <darren.powell@amd.com>
Fixes: 6db0c87a ("amdgpu/pm: Replace hwmgr smu usage of sprintf with sysfs_emit")
Bug: https://gitlab.freedesktop.org/drm/amd/-/issues/1774Reviewed-by: NLang Yu <lang.yu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

BOs need to be reserved before they are added or removed, so ensure that
they are reserved during kfd_mem_attach and kfd_mem_detach
Signed-off-by: NKent Russell <kent.russell@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]:
When we call hmm_range_fault to map memory after a migration, we don't
expect memory to be migrated again as a result of hmm_range_fault. The
driver ensures that all memory is in GPU-accessible locations so that
no migration should be needed. However, there is one corner case where
hmm_range_fault can unexpectedly cause a migration from DEVICE_PRIVATE
back to system memory due to a write-fault when a system memory page in
the same range was mapped read-only (e.g. COW). Ranges with individual
pages in different locations are usually the result of failed page
migrations (e.g. page lock contention). The unexpected migration back
to system memory causes a deadlock from recursive locking in our
driver.

[How]:
Creating a task reference new member under svm_range_list struct.
Setting this with "current" reference, right before the hmm_range_fault
is called. This member is checked against "current" reference at
svm_migrate_to_ram callback function. If equal, the migration will be
ignored.
Signed-off-by: NAlex Sierra <alex.sierra@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

There is no reason to allow for partial buffers from userspace in our
debugfs. In this particular case callers will zero out the wr_buf but if
callers in the future don't do that we might be looking at corrupt data.

Linus puts it better than I can in
https://lkml.org/lkml/2021/10/26/993Reviewed-by: NRodrigo Siqueira <Rodrigo.Siqueira@amd.com>
Signed-off-by: NHarry Wentland <harry.wentland@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NAric Cyr <aric.cyr@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NAnthony Koo <Anthony.Koo@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why & How]
In order to have dc_enable_dmub_notifications() more precise, add
one more condition to check if dc->debug.dpia_debug.bits.disable_dpia
is false.
Signed-off-by: NMeenakshikumar Somasundaram <meenakshikumar.somasundaram@amd.com>
Signed-off-by: NWayne Lin <Wayne.Lin@amd.com>
Signed-off-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
During Z10, root clock gating and memory low power registers needs to
to be restored if optimization is enabled in driver.

[How]
Added new DMUB boot option for root clock gating and memory low power.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NJake Wang <haonan.wang2@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why & How]
The MPC memory clocks should be powered down when not in use.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NJake Wang <haonan.wang2@amd.com>
Reviewed-by: NEric Yang <eric.yang2@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
HPO is only used for DP2.0. HPO HW control should be
disable when not being used to save power.

[How]
Shutdown HPO HW control during init hw.
Shutdown HPO HW control during stream disable.
Enable HPO HW control during stream enable if DP2.0.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NJake Wang <haonan.wang2@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why&how]
write LINK_SQUARE_PATTERN_num + 1 for square pulse pattern.
Specs requirement to write this register prior to write LINK_QUAL_LANEX_SET.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Reviewed-by: NGeorge Shen <george.shen@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
State can be cleared without removing individual streams (by
calling dc_remove_stream_from_ctx()). This can leave the
encoder assignment module in an incoherent state and cause
future assignments to be incorrect.

[How]
Clear encoder assignments when committing 0 streams or
re-initializing hardware.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NJimmy Kizito <Jimmy.Kizito@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
In scenario when 1 display connected with pipe split (2 pipes in use)
and 3 new displays simultaneously hotplugged via MST hub (4 pipes in use),
mpcc may get reprogram to other vtg, remaining busy.
In this case waiting for mpcc idle timeouts with error like this:
[drm] REG_WAIT timeout 1us * 100000 tries - mpc2_assert_idle_mpcc
RIP: 0010:mpc2_assert_mpcc_idle_before_connect
Call Trace:
dcn20_update_mpcc
dcn20_program_front_end_for_ctx
dc_commit_state
amdgpu_dm_atomic_commit_tail
...

[How]
Add pipe split change condition to disable dangling plane.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NRoman Li <Roman.Li@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Reviewed-by: NDmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
1. YCbCr 4:2:2 8bpc/10bpc modes are blocked for HDMI by policy
2. A YCbCr 4:2:0 calculation error blocked some 4:2:0 timing modes

[How]
YCbCr 4:2:2 8bpc/10bpc modes are allowed for HDMI
Fix YCbCr 4:2:0 calculation error
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NBing Guo <Bing.Guo@amd.com>
Reviewed-by: NChris Park <chris.park@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
MST monitor sends link loss short pulse continuous but sink is
occupy by HDMI input to lead link training fail.

[How]
disable link once retraining fail.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NYu-ting Shen <yu-tshen@amd.com>
Reviewed-by: NWenjing Liu <Wenjing.Liu@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
It was found that the system would hang on a dummy pstate when playing
4k60 videos on a 1080p 390Hz monitor.

[HOW]
Properly select the dummy_pstate_latency_ms when firmware assisted
memory clock switching is enabled instead of assuming that the highest
latency would work for every monitor timing.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NFelipe Clark <felclark@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Fix spacing issue for the format string.

Addresses-Coverity-ID: 1446765: ("Invalid printf format string")
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NAnson Jacob <Anson.Jacob@amd.com>
Reviewed-by: NRodrigo Siqueira <Rodrigo.Siqueira@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
If timing and bpp of displays on mst hub are not changed,
pbn, slot_num for displays should not be changed. Linux
user mode may initiate atomic_check with different display
configuration after set mode finished. This will call to
amdgpu_dm to re-compute payload, slot_num of displays and
saved to dm_connect_state. stream->timing.flags.dsc, pbn,
slot_num are updated to values which may be different from
that were used for set mode. when dsc hub with 3 4k@60hz dp
connected, 3 dsc engines are enabled. timing.flags.dsc = 1.
timing.flags.dsc are changed to 0 due to atomic check. when
dsc hub is unplugged, amdgpu driver check timing.flags.dsc
for last mode set and find out flags.dsc = 0, then does not
disable dsc.

[HOW]
check status of  displays on dsc mst hubs. re-compute pbn,
slot_num, timing.flags.dsc only if there is mode, connect
or enable/disable change.
Acked-by: NAurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: NHersen Wu <hersenwu@amd.com>
Reviewed-by: NMikita Lipski <Mikita.Lipski@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Remove duplicated kfd_resume_iommu which already runs
in mdgpu_amdkfd_device_init.
Tested-By: NKen Moffat <zarniwhoop@ntlworld.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For yellow carp, the desired CGPG hysteresis value is 0x4E20.
Signed-off-by: NAaron Liu <aaron.liu@amd.com>
Acked-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

commit 652de07a ("drm/amd/display: Fully switch to dmub for all dcn21
asics") switched over to using dmub on Renoir to fix Gitlab 1735, but this
implied a new dependency on newer firmware which might not be met on older
kernel versions.

Since sw_init runs before hw_init, there is an opportunity to determine
whether or not the firmware version is new to adjust the behavior.

Cc: Roman.Li@amd.com
BugLink: https://gitlab.freedesktop.org/drm/amd/-/issues/1772
BugLink: https://gitlab.freedesktop.org/drm/amd/-/issues/1735
Fixes: 652de07a ("drm/amd/display: Fully switch to dmub for all dcn21 asics")
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NRoman Li <Roman.Li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For ASICs not supporting power profile mode, don't show the attribute.
Verify that the function has been implemented by the subsystem.
Suggested-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This better aligns that the caller can make a mistake with the buffer
and -EINVAL should be returned, but if the hardware doesn't support
the feature it should be -EOPNOTSUPP.
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Prevent possible issues from set and get being called simultaneously.
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This was added by commit bd8dcea9 ("drm/amd/pm: add callbacks to
read/write sysfs file pp_power_profile_mode") but the feature was
deprecated from PMFW.  Remove it from the driver.
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Previously Renoir compiler gfx target version was forced to Raven.
Update driver side for completeness.
Signed-off-by: NGraham Sider <Graham.Sider@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is more useful when talking to the SMU team to have the information
in this format, save one less step to manually do it.
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>