Only select HMM_MIRROR will get kernel config dependency warnings
if CONFIG_HMM is missing in the config. Add depends on HMM will
solve the issue.

Add conditional compilation to fix compilation errors if HMM_MIRROR
is not enabled as HMM config is not enabled.

Remove unused function amdgpu_ttm_tt_mark_user_pages.
Signed-off-by: NPhilip Yang <Philip.Yang@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

HMM simplify the CPU page table concurrent update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM does not pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().
Signed-off-by: NPhilip Yang <Philip.Yang@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

There is circular lock between gfx and kfd path with HMM change:
lock(dqm) -> bo::reserve -> amdgpu_mn_lock

To avoid this, move init/unint_mqd() out of lock(dqm), to remove nested
locking between mmap_sem and bo::reserve. The locking order
is: bo::reserve -> amdgpu_mn_lock(p->mn)
Signed-off-by: NPhilip Yang <Philip.Yang@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Replace our MMU notifier with hmm_mirror_ops.sync_cpu_device_pagetables
callback. Enable CONFIG_HMM and CONFIG_HMM_MIRROR as a dependency in
DRM_AMDGPU_USERPTR Kconfig.

It supports both KFD userptr and gfx userptr paths.
Signed-off-by: NPhilip Yang <Philip.Yang@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Use BACO for reset of the platform supports it.
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Several of the BACO functions are common across smu9-based
asics.  Split the common code out.
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This implements BACO (Bus Active, Chip Off) support
for vega12.
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NMark McGarrity <mark.mcgarrity@amd.com>
Reviewed-by: NTony Cheng <Tony.Cheng@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add a dc_bounding_box_overrides struct to define bb overrides. It is
loaded in during DC init.
Signed-off-by: NJun Lei <Jun.Lei@amd.com>
Reviewed-by: NTony Cheng <Tony.Cheng@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
dc_commit_updates_for_stream is called twice per stream: once
with the flip data and once will all other data. This causes
problems when these DC calls have different numbers of planes

For example, a commit with a pageflip on plane A and a
non-pageflip change on plane B will first call
into DC with just plane A, causing plane B to be
disabled. Then it will call into DC with both planes,
re-enabling plane B

[How]
Merge flip and full into a single bundle

Apart from the single DC call, the logic should not be
changed by this patch
Signed-off-by: NDavid Francis <David.Francis@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
eDP v1.4 allows panels to report link rates other than RBR/HBR/HBR2, that
may be more optimal for the panel's timing. Power can be saved by using
a link rate closer to the required bandwidth of the panel's timing.

[How]
Scan the table of reported link rates from the panel, and select the
minimum link rate that satisfies the bandwidth requirements of the panel's
timing. Include a flag to make the feature optional.
Signed-off-by: NJosip Pavic <Josip.Pavic@amd.com>
Reviewed-by: NHarry Wentland <Harry.Wentland@amd.com>
Acked-by: NAnthony Koo <Anthony.Koo@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Will be used to signify if P-state change is supported.
Signed-off-by: NJun Lei <Jun.Lei@amd.com>
Reviewed-by: NEric Yang <eric.yang2@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
The wait_for_vblank boolean in commit_tail was passed by reference
into each stream commit, and if that commit was an asynchronous
flip, it would disable vblank waits on all subsequent flips.

This made the behaviour depend on crtc order in a non-intuitive way,
although since the asynchronous pageflip flag is only used by the
legacy IOCTLs at the moment it is never an issue

[How]
Find wait_for_vblank before doing any stream commits
Signed-off-by: NDavid Francis <David.Francis@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[WHY]
When doing split pipe, if one pipe is pending on flip, the entire
plane's status should be flip pending, otherwise corruption can occur
when OS writes to a surface prematurely.

[HOW]
Clear the flip pending bit before checking pipes, then OR the flip
pending bits from all pipes together to create the flip pending status
of the entire plane.
Signed-off-by: NWesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NEryk Brol <Eryk.Brol@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
AUX arbitration occurs between SW and FW components.
When AUX acquire fails, it causes engine->ddc to be NULL,
which leads to an exception when we try to release the AUX
engine.

[How]
When AUX engine acquire fails, it should return from the
function without trying to continue the operation.
The upper level will determine if it wants to retry.
i.e. dce_aux_transfer_with_retries will be used and retry.
Signed-off-by: NAnthony Koo <Anthony.Koo@amd.com>
Reviewed-by: NAric Cyr <Aric.Cyr@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
Seamless boot tries to reuse planes that were enabled for the first
commit applied.

In the case where Raven is booting with two monitors connected and the
first commit contains two streams the screen corruption would occur
because the second stream was trying to re-use a tg and plane that
weren't previously enabled.

The state on the first commit looks something like the following:

TG0: enabled=1
TG1: enabled=0
TG2: enabled=0
TG3: enabled=0

New state: pipe=0, stream=0,    plane=0,       new_tg=0
New state: pipe=1, stream=1,    plane=1,       new_tg=1
New state: pipe=2, stream=NULL, plane=NULL,    new_tg=NULL
New state: pipe=3, stream=NULL, plane=NULL,    new_tg=NULL

Only one plane/tg is setup before we enter accelerated mode so
we really want to disabling everything but that first plane.

[How]

Check if the stream is not NULL and if the tg is enabled before
deciding whether to skip the plane disable.

Also ensure we're also disabling on the current state's pipe_ctx so
we don't overwrite the fields in the new pending state.
Signed-off-by: NNicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Reviewed-by: NAnthony Koo <Anthony.Koo@amd.com>
Acked-by: NHarry Wentland <Harry.Wentland@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This change changes dc add plane logic to allow plane-less resource
reservation (pipe split).

If a free pipe_ctx (no plane_state attached) is the head pipe, and is
found with a bottom pipe attached, assign the plane to add on the bottom
pipe.

In addition, prepend dcn10 to dcn10-specific reset_back_end_for_pipe
and reset_hw_ctx_wrap
Signed-off-by: NDmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Reviewed-by: NCharlene Liu <Charlene.Liu@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Driver need to  poll the SURFACE_INUSE register to determine when to
start the new task and write data to the checked surface.

Implement the wait functions, and add the necessary hubbub registers.
Signed-off-by: NCharlene Liu <charlene.liu@amd.com>
Reviewed-by: NDmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
For later use by the DC VM implementation
Signed-off-by: NEryk Brol <eryk.brol@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
Seamless boot code not working on all ASICs because of
some underflow issues caused by some uninitialized HW
state.

[How]
Keep some logical and power gating init code in hw_init.
Move some per pipe init code to enable accelerated mode
Signed-off-by: NAnthony Koo <Anthony.Koo@amd.com>
Reviewed-by: NAric Cyr <Aric.Cyr@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
DM has impelemented new pp_smu interface. 'Else' is not longer needed.
Signed-off-by: NFatemeh Darbehani <fatemeh.darbehani@amd.com>
Reviewed-by: NEric Yang <eric.yang2@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Acked-by: NYongqiang Sun <yongqiang.sun@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This enum definition is shared, so move it to a shared location.
Signed-off-by: NEric Bernstein <eric.bernstein@amd.com>
Reviewed-by: NTony Cheng <Tony.Cheng@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: Nmmcgarri <mark.mcgarrity@amd.com>
Reviewed-by: NTony Cheng <Tony.Cheng@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
Make dml and integration with pplib clearer.

[How]
Change the way the dml formula is initialized to make its values more
clear. Restructure DC interface with pplib into rv_funcs.
Cap clocks received from pplib.
Signed-off-by: NJun Lei <Jun.Lei@amd.com>
Signed-off-by: NEryk Brol <eryk.brol@amd.com>
Reviewed-by: NDmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Added a "disable_tri_buf" DC debug option. When set to 1  feature will
be off.
Signed-off-by: NCharlene Liu <charlene.liu@amd.com>
Reviewed-by: NDmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Acked-by: NLeo Li <sunpeng.li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

In VRR mode, keep track of the vblank count of the last
completed pageflip in amdgpu_crtc->last_flip_vblank, as
recorded in the pageflip completion handler after each
completed flip.

Use that count to prevent mmio programming a new pageflip
within the same vblank in which the last pageflip completed,
iow. to throttle pageflips to at most one flip per video
frame, while at the same time allowing to request a flip
not only before start of vblank, but also anywhere within
vblank.

The old logic did the same, and made sense for regular fixed
refresh rate flipping, but in vrr mode it prevents requesting
a flip anywhere inside the possibly huge vblank, thereby
reducing framerate in vrr mode instead of improving it, by
delaying a slightly delayed flip requests up to a maximum
vblank duration + 1 scanout duration. This would limit VRR
usefulness to only help applications with a very high GPU
demand, which can submit the flip request before start of
vblank, but then have to wait long for fences to complete.

With this method a flip can be both requested and - after
fences have completed - executed, ie. it doesn't matter if
the request (amdgpu_dm_do_flip()) gets delayed until deep
into the extended vblank due to cpu execution delays. This
also allows clients which want to regulate framerate within
the vrr range a much more fine-grained control of flip timing,
a feature that might be useful for video playback, and is
very useful for neuroscience/vision research applications.

In regular non-VRR mode, retain the old flip submission
behavior. This to keep flip scheduling for fullscreen X11/GLX
OpenGL clients intact, if they use the GLX_OML_sync_control
extensions glXSwapBufferMscOML(, ..., target_msc,...) function
with a specific target_msc target vblank count.

glXSwapBuffersMscOML() or DRI3/Present PresentPixmap() will
not flip at the proper target_msc for a non-zero target_msc
if VRR mode is active with this patch. They'd often flip one
frame too early. However, this limitation should not matter
much in VRR mode, as scheduling based on vblank counts is
pretty futile/unusable under variable refresh duration
anyway, so no real extra harm is done.

According to some testing already done with this patch by
Nicholas on top of my tests, IGT tests didn't report any
problems. If fixes stuttering and flickering when flipping
at rates below the minimum vrr refresh rate.

Fixes: bb47de73 ("drm/amdgpu: Set FreeSync state using drm VRR
properties")
Signed-off-by: NMario Kleiner <mario.kleiner.de@gmail.com>
Cc: <stable@vger.kernel.org>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Michel Dänzer <michel@daenzer.net>
Tested-by: NBruno Filipe <bmilreu@gmail.com>
Reviewed-by: NNicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Signed-off-by: NNicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Clear the VM PDs/PTs only after initializing all the structures.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Clang warns when an expression that equals zero is used as a null
pointer constant (in lieu of NULL):

drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm.c:4435:3:
warning: expression which evaluates to zero treated as a null pointer
constant of type 'const enum color_transfer_func *'
[-Wnon-literal-null-conversion]
                TRANSFER_FUNC_UNKNOWN,
                ^~~~~~~~~~~~~~~~~~~~~
1 warning generated.

This warning is caused by commit bb47de73 ("drm/amdgpu: Set FreeSync
state using drm VRR properties") and it could be solved by using NULL
instead of TRANSFER_FUNC_UNKNOWN or casting TRANSFER_FUNC_UNKNOWN as a
pointer. However, after looking into it, there doesn't appear to be a
good reason to pass app_tf by reference as it is never mutated along the
way. This is the only code path in which app_tf is used:

mod_freesync_build_vrr_infopacket ->
    build_vrr_infopacket_v2 ->
        build_vrr_infopacket_fs2_data

Neither mod_freesync_build_vrr_infopacket or build_vrr_infopacket_v2
modify app_tf's value and build_vrr_infopacket_fs2_data expects just
the value so we can avoid dereferencing anything by just passing in
app_tf's value to mod_freesync_build_vrr_infopacket and
build_vrr_infopacket_v2.

There is no functional change because build_vrr_infopacket_fs2_data
doesn't do anything if TRANSFER_FUNC_UNKNOWN is passed to it, the same
as not calling build_vrr_infopacket_fs2_data at all like before this
change when NULL was used for app_tf.
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Signed-off-by: NNathan Chancellor <natechancellor@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This reverts commit 2172b89e.
Signed-off-by: NCandice Li <candice.li@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of the hard-coded ones from VBIOS.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Set the soft/hard max settings as max possible to
not violate the OD settings.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Set the default fclk as what we got from VBIOS.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

There will be some needs to retrieve clock information from other
sysplls also except default 0.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For UCLK_FMAX OD feature, SMU overwrites the highest UCLK DPM level freq.
Therefore it can only take values that are greater than the second highest
DPM level freq.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This can fix possible screen freeze on high resolution displays.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

As these settings got reset during above phm_apply_clock_adjust_rules.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

As these are already set during apply_clocks_adjust_rules.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

if use the legacy method to allocate object, when mqd_hiq need to run
uninit code, it will be cause WARNING call trace.

eg: (s3 suspend test)
[   34.918944] Call Trace:
[   34.918948]  [<ffffffff92961dc1>] dump_stack+0x19/0x1b
[   34.918950]  [<ffffffff92297648>] __warn+0xd8/0x100
[   34.918951]  [<ffffffff9229778d>] warn_slowpath_null+0x1d/0x20
[   34.918991]  [<ffffffffc03ce1fe>] uninit_mqd_hiq_sdma+0x4e/0x50 [amdgpu]
[   34.919028]  [<ffffffffc03d0ef7>] uninitialize+0x37/0xe0 [amdgpu]
[   34.919064]  [<ffffffffc03d15a6>] kernel_queue_uninit+0x16/0x30 [amdgpu]
[   34.919086]  [<ffffffffc03d26c2>] pm_uninit+0x12/0x20 [amdgpu]
[   34.919107]  [<ffffffffc03d4915>] stop_nocpsch+0x15/0x20 [amdgpu]
[   34.919129]  [<ffffffffc03c1dce>] kgd2kfd_suspend.part.4+0x2e/0x50 [amdgpu]
[   34.919150]  [<ffffffffc03c2667>] kgd2kfd_suspend+0x17/0x20 [amdgpu]
[   34.919171]  [<ffffffffc03c103a>] amdgpu_amdkfd_suspend+0x1a/0x20 [amdgpu]
[   34.919187]  [<ffffffffc02ec428>] amdgpu_device_suspend+0x88/0x3a0 [amdgpu]
[   34.919189]  [<ffffffff922e22cf>] ? enqueue_entity+0x2ef/0xbe0
[   34.919205]  [<ffffffffc02e8220>] amdgpu_pmops_suspend+0x20/0x30 [amdgpu]
[   34.919207]  [<ffffffff925c56ff>] pci_pm_suspend+0x6f/0x150
[   34.919208]  [<ffffffff925c5690>] ? pci_pm_freeze+0xf0/0xf0
[   34.919210]  [<ffffffff926b45c6>] dpm_run_callback+0x46/0x90
[   34.919212]  [<ffffffff926b49db>] __device_suspend+0xfb/0x2a0
[   34.919213]  [<ffffffff926b4b9f>] async_suspend+0x1f/0xa0
[   34.919214]  [<ffffffff922c918f>] async_run_entry_fn+0x3f/0x130
[   34.919216]  [<ffffffff922b9d4f>] process_one_work+0x17f/0x440
[   34.919217]  [<ffffffff922bade6>] worker_thread+0x126/0x3c0
[   34.919218]  [<ffffffff922bacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[   34.919220]  [<ffffffff922c1c31>] kthread+0xd1/0xe0
[   34.919221]  [<ffffffff922c1b60>] ? insert_kthread_work+0x40/0x40
[   34.919222]  [<ffffffff92974c1d>] ret_from_fork_nospec_begin+0x7/0x21
[   34.919224]  [<ffffffff922c1b60>] ? insert_kthread_work+0x40/0x40
[   34.919224] ---[ end trace 38cd9f65c963adad ]---
Signed-off-by: NKevin Wang <kevin1.wang@amd.com>
Reviewed-by: NOak Zeng <Oak.Zeng@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This patch uses REG32_PCIE wrapper instead of writting pci_index2 and reading
pci_data2 for psp. This sequence should be protected by pcie_idx_lock.
Signed-off-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This patch uses REG32_PCIE wrapper instead of writting pci_index2 and reading
pci_data2 for powerplay. This sequence should be protected by pcie_idx_lock.
Signed-off-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>