Calling free_mqd inside of destroy_queue_nocpsch_locked can cause a
circular lock. destroy_queue_nocpsch_locked is called under a DQM lock,
which is taken in MMU notifiers, potentially in FS reclaim context.
Taking another lock, which is BO reservation lock from free_mqd, while
causing an FS reclaim inside the DQM lock creates a problematic circular
lock dependency. Therefore move free_mqd out of
destroy_queue_nocpsch_locked and call it after unlocking DQM.
Signed-off-by: NAmber Lin <Amber.Lin@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This should fix a kernel LOCKDEP warning on Vega10:
[  149.416604] ================================
[  149.420877] WARNING: inconsistent lock state
[  149.425152] 5.11.0-kfd-fkuehlin #517 Not tainted
[  149.429770] --------------------------------
[  149.434053] inconsistent {HARDIRQ-ON-W} -> {IN-HARDIRQ-W} usage.
[  149.440059] swapper/3/0 [HC1[1]:SC0[0]:HE0:SE1] takes:
[  149.445198] ffff9ac80e005d68 (&adev->vm_manager.pasid_lock){?.+.}-{2:2}, at: amdgpu_vm_get_task_info+0x25/0x90 [amdgpu]
[  149.456252] {HARDIRQ-ON-W} state was registered at:
[  149.461136]   lock_acquire+0x242/0x390
[  149.464895]   _raw_spin_lock+0x2c/0x40
[  149.468647]   amdgpu_vm_handle_fault+0x44/0x380 [amdgpu]
[  149.474187]   gmc_v9_0_process_interrupt+0xa8/0x410 [amdgpu]
...
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NOak Zeng <Oak.Zeng@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

If GC has entered CGPG, ringing doorbell > first page doesn't wakeup GC.
Enlarge CP_MEC_DOORBELL_RANGE_UPPER to workaround this issue.
Signed-off-by: NYifan Zhang <yifan1.zhang@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

If GC has entered CGPG, ringing doorbell > first page doesn't wakeup GC.
Enlarge CP_MEC_DOORBELL_RANGE_UPPER to workaround this issue.
Signed-off-by: NYifan Zhang <yifan1.zhang@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Move shadow_list to struct amdgpu_bo_vm as shadow BOs
are part of PT/PD BOs.
Signed-off-by: NNirmoy Das <nirmoy.das@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Make provision to pass different ttm BO destroy callback
while creating a amdgpu_bo.
Signed-off-by: NNirmoy Das <nirmoy.das@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Page table entries are now in embedded in VM BO, so
we do not need struct amdgpu_vm_pt. This patch replaces
struct amdgpu_vm_pt with struct amdgpu_vm_bo_base.
Signed-off-by: NNirmoy Das <nirmoy.das@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For ASICs that need to load sys_drv_aux and sos_aux,
the sys_start_addr is not the start address of psp
ucode array because the sys_drv_aux and sos_aux actaully
located at the end of the ucode array, instead, the
psp ucode arrary start address should be sos_hdr +
sos_hdr_offset.
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NJohn Clements <John.Clements@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When some GPUs don't support SVM, don't disabe it for the entire process.
That would be inconsistent with the information the process got from the
topology, which indicates SVM support per GPU.

Instead disable SVM support only for the unsupported GPUs. This is done
by checking any per-device attributes against the bitmap of supported
GPUs. Also use the supported GPU bitmap to initialize access bitmaps for
new SVM address ranges.

Don't handle recoverable page faults from unsupported GPUs. (I don't
think there will be unsupported GPUs that can generate recoverable page
faults. But better safe than sorry.)
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NPhilip Yang <philip.yang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
DP LL Compliance tests require that the first DPCD transactions after a
hotplug have a timeout interval of 3.2 ms.  In cases where LTTPR is
disabled, this means that the first reads from DP_SET_POWER and DP_DPCD_REV must have an extended
timeout.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
When changing the DISPCLK_WDIVIDER value from 126 to 127, the change in
clock rate is too great for the FIFOs to handle. This can cause visible
corruption during clock change.

HW has handed down this register sequence to fix the issue.

[HOW]
The sequence, from HW:
a.	127 -> 126
Read  DIG_FIFO_CAL_AVERAGE_LEVEL
FIFO level N = DIG_FIFO_CAL_AVERAGE_LEVEL / 4
Set DCCG_FIFO_ERRDET_OVR_EN = 1
Write 1 to OTGx_DROP_PIXEL for (N-4) times
Set DCCG_FIFO_ERRDET_OVR_EN = 0
Write DENTIST_DISPCLK_RDIVIDER = 126

Because of frequency stepping, sequence a can be executed to change the
divider from 127 to any other divider value.

b.	126 -> 127
Read  DIG_FIFO_CAL_AVERAGE_LEVEL
FIFO level N = DIG_FIFO_CAL_AVERAGE_LEVEL / 4
Set DCCG_FIFO_ERRDET_OVR_EN = 1
Write 1 to OTGx_ADD_PIXEL for (12-N) times
Set DCCG_FIFO_ERRDET_OVR_EN = 0
Write DENTIST_DISPCLK_RDIVIDER = 127

Because of frequency stepping, divider must first be set from any other
divider value to 126 before executing sequence b.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NDmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
Hardware has handed down a new sequence requiring the value of this
register be read from clk_mgr.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NDmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
SCR for DP 2.0 spec says that multiple LTTPRs must not be accessed in a
single AUX transaction.
There may be other places in future where breaking up AUX accesses is
necessary.

[HOW]
Partition the entire DPCD address space into blocks. When an incoming AUX
request spans multiple blocks, break up the request into multiple requests.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
DP 2.0 SCR specifies that
"A DPTX shall distinguish I2C_DEFER|AUX_ACK from AUX_DEFER. AUX retries
due to
I2C_DEFER are not counted as part of minimum 7 retires (sic) upon
AUX_DEFER’s"
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
DP 2.0 SCR specifies that TX devices must retry at least 7 times when
receiving an AUX DEFER reply from RX. In addition, the specification
states that the TX shall not retry indefinitely, and gives a suggestive
timeout interval of 50ms.

[HOW]
Keep retrying until both 7 or more retries have been made, and the 50ms
interval has passed.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
7 is the minimum number of retries TX must attempt on an AUX DEFER, not
the maximum.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
Some DPCD addresses, notably LTTPR Capability registers, are expected to
be read all together in a single DPCD transaction. Rather than force callers to
read registers they don't need, we want to quietly extend the addresses
read, and only return back the values the caller asked for.
This does not affect DPCD writes.

[HOW]
Create an additional layer above AUX to perform 'checked' DPCD
transactions.
Iterate through an array of DPCD address ranges that are marked as being
contiguous. If a requested read falls within one of those ranges, extend
the read to include the entire range.
After DPCD has been queried, copy the requested bytes into the caller's
data buffer, and deallocate all resources used.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
DP specifies that an LTTPR device is only present if PHY_REPEATER_CNT is
0x80, 0x40, 0x20, 0x10, 0x08, 0x04, 0x02, or 0x01.

All other values should be considered no LTTPRs present.

[HOW]
Function dp_convert_to_count already does this check. Use it to determine
if PHY_REPEATER_CNT is a valid LTTPR count.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
SCR for DP2.0 requires that LT be performed with PHY_REPEATER_MODE
programmed to 0x55 (Transparent) whenever PHY_REPEATER_CNT is any value
other than 0x80, 0x40, 0x20, 0x10, 0x08, 0x04, 0x02, or 0x01.

[HOW]
Write Non-Transparent (0xAA) to PHY_REPEATER_MODE when LTTPRs detected and Non-Transparent is
requested.
Write Transparent in all other cases.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
SCR for DP 2.0 Spec states that a DPTX shall put LTTPRs into Transparent
mode after reading LTTPR Capability registers on HPD.

The wording of the SCR is somewhat ambiguous as to whether
Transparent mode must be set explicity, or is implicitly set on LTTPR
capability read. Explicitly setting Transparent mode after LTTPR
capability read should cover all
cases.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
SCR for DP 2.0 requires that LTTPR caps be read first on hotplug.
For the sake of consistency, this should also be the case on bootup.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
We want LTTPR capabilities to be readable from more places than just
retrieve_link_cap
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
A new SCR for the DP2.0 spec requires that LTTPR caps be the first thing
read from DPCD upon hotplug.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
Improve the maintain/read abilities of dm code.

[How]
Create amdgpu_dm_psr.c/h files.
Move psr function from amdgpu_dm.c
Signed-off-by: NRoman Li <roman.li@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This version brings along following fixed:
   - LTTPR improvements
   - Backlight improvements
   - eDP hotplug detection
Signed-off-by: NAric Cyr <aric.cyr@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAnthony Koo <Anthony.Koo@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
Fixed spelling error.

[How]
Changed "currnet_setting" to "current_setting".
Signed-off-by: NDavid Galiffi <David.Galiffi@amd.com>
Reviewed-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
To change the swizzle visual confirm reference pipe from top pipe to
bottom pipe due to bottom pipe information would be more important
for multiple overlay case.
Signed-off-by: NPo-Ting Chen <robin.chen@amd.com>
Reviewed-by: NAnthony Koo <Anthony.Koo@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Move mst start top mgr in dc_link_detect layer.
Remove unused same_dpcd variable.
Move PEAK_FACTOR_X1000 and LINK_TRAINING_MAX_VERIFY_RETRY
to the proper header for defining dc link internal constant.
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Reviewed-by: NGeorge Shen <George.Shen@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why & How]
Tune backlight ramping profiles for each Vari-Bright level to suit
customer preferences
Signed-off-by: NJosip Pavic <Josip.Pavic@amd.com>
Reviewed-by: NAnthony Koo <Anthony.Koo@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
Some custom platforms use eDP hotplug events to notify panel
capability changes that should be reported

[How]
Add a DC config option that unblocks eDP hotplug events
Signed-off-by: NYi-Ling Chen <Yi-Ling.Chen2@amd.com>
Reviewed-by: NAric Cyr <Aric.Cyr@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
Ability to triage DMCUB is improved with availability of certain
dmub registers not currently captured in crash dump diagnostic data.

[HOW]
Add dmub registers to diagnostic data collection.

Thanks Nicholas Kazlauskas for awesome input on this!
Signed-off-by: NAshley Thomas <Ashley.Thomas2@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

cleanup
Signed-off-by: NDmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Reviewed-by: NAric Cyr <Aric.Cyr@amd.com>
Acked-by: NAnson Jacob <Anson.Jacob@amd.com>
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

get_wave_state acquires the mmap_lock on copy_to_user but so do
mmu_notifiers.  mmu_notifiers allows dqm locking so do get_wave_state
outside the dqm_lock to prevent circular locking.

v2: squash in unused variable removal.
Signed-off-by: NJonathan Kim <jonathan.kim@amd.com>
Reviewed-by: NFelix Kuehling <felix.kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For each CRTC state, check the size of Gamma and Degamma LUTs  so
unexpected and larger sizes wouldn't slip through.

TEST: IGT:kms_color::pipe-invalid-gamma-lut-sizes

v2: fix assignments in if clauses, Mark's email.
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Signed-off-by: NMark Yacoub <markyacoub@chromium.org>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

GECC and G6 mem training are mutually exclusive
functionalities. VBIOS/PSP will set the flag
(BOOT_CFG_FEATURE_TWO_STAGE_DRAM_TRAINING) in
runtime database to indicate whether dram memory
training need to be disabled or not.

For Navi1x families, two stage mem training is always
enabled.
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NJohn Clements <john.clements@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

PSP runtime boot_cfg_bitmask carries various psp bl
feature bit mask that can be used by driver. Cache
it in sw_init for further usage.
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NJohn Clements <john.clements@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

PSP will dump various boot up information into a
portion of local frame buffer, called runtime database.
The helper function is used for driver to query those
shared information.

v2: init ret and check !ret to exit loop as soon as
found the entry
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NJohn Clements <john.clements@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

PSP runtime database is used to share various
boot up information with driver.
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NJohn Clements <john.clements@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Dynamic GECC allows user to specify GECC enablement
status, which will take effect in next boot cycle.

v2: initialize boot_cfg to 0xFF
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NJohn Clements <john.clements@amd.com>
Reviewed-by: NLijo Lazar <lijo.lazar@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>