Checkpoint contents of queue control stacks on CRIU dump and restore them
during CRIU restore.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Checkpoint contents of queue MQD's on CRIU dump and restore them during
CRIU restore.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When re-creating queues during CRIU restore, restore the queue with the
same doorbell id value used during CRIU dump.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When re-creating queues during CRIU restore, restore the queue with the
same sdma id value used during CRIU dump.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When re-creating queues during CRIU restore, restore the queue with the
same queue id value used during CRIU dump.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add support to existing CRIU ioctl's to save number of queues and queue
properties for each queue during checkpoint and re-create queues on
restore.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Introducing UNPAUSE op. After CRIU amdgpu plugin performs a PROCESS_INFO
op the queues will be stay in an evicted state. Once the plugin is done
draining BO contents, it is safe to perform an UNPAUSE op for the queues
to resume.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This adds support to create userptr BOs on restore and introduces a new
ioctl op to restart memory notifiers for the restored userptr BOs.
When doing CRIU restore MMU notifications can happen anytime after we call
amdgpu_mn_register. Prevent MMU notifications until we reach stage-4 of the
restore process i.e. criu_resume ioctl op is received, and the process is
ready to be resumed. This ioctl is different from other KFD CRIU ioctls
since its called by CRIU master restore process for all the target
processes being resumed by CRIU.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This implements the KFD CRIU Restore ioctl that lays the basic
foundation for the CRIU restore operation. It provides support to
create the buffer objects corresponding to the checkpointed image.
This ioctl creates various types of buffer objects such as VRAM,
MMIO, Doorbell, GTT based on the date sent from the userspace plugin.
The data mostly contains the previously checkpointed KFD images from
some KFD processs.

While restoring a criu process, attach old IDR values to newly
created BOs. This also adds the minimal gpu mapping support for a single
gpu checkpoint restore use case.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This adds support to discover the  buffer objects that belong to a
process being checkpointed. The data corresponding to these buffer
objects is returned to user space plugin running under criu master
context which then stores this info to recreate these buffer objects
during a restore operation.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This IOCTL op is expected to be called as a precursor to the actual
Checkpoint operation. This does the basic discovery into the target
process seized by CRIU and relays the information to the userspace that
utilizes it to start the Checkpoint operation via another dedicated
IOCTL op.

The process_info IOCTL op determines the number of GPUs, buffer objects
that are associated with the target process, its process id in
caller's namespace since /proc/pid/mem interface maybe used to drain
the contents of the discovered buffer objects in userspace and getpid
returns the pid of CRIU dumper process. Also the pid of a process
inside a container might be different than its global pid so return
the ns pid.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Checkpoint-Restore in userspace (CRIU) is a powerful tool that can
snapshot a running process and later restore it on same or a remote
machine but expects the processes that have a device file (e.g. GPU)
associated with them, provide necessary driver support to assist CRIU
and its extensible plugin interface. Thus, In order to support the
Checkpoint-Restore of any ROCm process, the AMD Radeon Open Compute
Kernel driver, needs to provide a set of new APIs that provide
necessary VRAM metadata and its contents to a userspace component
(CRIU plugin) that can store it in form of image files.

This introduces some new ioctls which will be used to checkpoint-Restore
any KFD bound user process. KFD only allows ioctl calls from the same
process that opened the KFD file descriptor. Since these ioctls are
expected to be called from a KFD criu plugin which has elevated ptrace
attached privileges and CAP_CHECKPOINT_RESTORE capabilities attached with
the file descriptors so modify KFD to allow such calls.

(API redesigned by David Yat Sin)
Suggested-by: NFelix Kuehling <felix.kuehling@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

MESA polls for errors every 2-3 seconds. Printing with dev_info() causes
the dmesg log to fill up with the same message, e.g,

[18028.206676] amdgpu 0000:0b:00.0: amdgpu: df doesn't config ras function.

Make it dev_dbg_once(), as it isn't something correctible during boot or
thereafter, so printing just once is sufficient. Also sanitize the message.

Cc: Alex Deucher <Alexander.Deucher@amd.com>
Cc: Hawking Zhang <Hawking.Zhang@amd.com>
Cc: John Clements <john.clements@amd.com>
Cc: Tao Zhou <tao.zhou1@amd.com>
Cc: yipechai <YiPeng.Chai@amd.com>
Fixes: 8b0fb0e9 ("drm/amdgpu: Modify gfx block to fit for the unified ras block data and ops")
Signed-off-by: NLuben Tuikov <luben.tuikov@amd.com>
Reviewed-by: NAlex Deucher <Alexander.Deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Some people complained about the name and this matches much
more Linux naming conventions for object functions.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Whenever a bo_va structure is added or removed the VM and eventually
added BO should be locked.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Assigning 0L to a pointer variable caused the following warning:

drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dsc/rc_calc_fpu.c:71:40:
warning: Using plain integer as NULL pointer

In order to remove this warning, this commit assigns a NULL pointer to
the pointer variable that caused this issue.
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NMagali Lemes <magalilemes00@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

   (v3)
     Rewrote patchset to order patches as (API, hw impl, usecase)

     - added API for new power management function emit_clk_levels
       This function should duplicate the functionality of print_clk_levels,
       but this solution passes the buffer base and write offset down the stack.
     - new powerplay function emit_clock_levels, implemented by smu_emit_ppclk_levels()
       This function parallels the implementation of smu_print_ppclk_levels and
       calls emit_clk_levels, and allows the returns of errors
     - new helper function smu_convert_to_smuclk called by smu_print_ppclk_levels and
       smu_emit_ppclk_levels
Signed-off-by: NDarren Powell <darren.powell@amd.com>
Reviewed-By: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

trace_amdgpu_vm_update_ptes trace unable to log when nptes too large
Signed-off-by: NSomalapuram Amaranath <Amaranath.Somalapuram@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

dGPUs connected to Intel systems configured for suspend to idle
will not have the power rails cut at suspend and resetting the GPU
may lead to problematic behaviors.

Fixes: e25443d2 ("drm/amdgpu: add a dev_pm_ops prepare callback (v2)")
Link: https://gitlab.freedesktop.org/drm/amd/-/issues/1879Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We ran into the problem that clearing really larger buffer (60GiB) caused an
SDMA timeout.

Restructure the function to use the dst window instead of mapping the whole
buffer into the GART and then fill only 2MiB/256MiB chunks at a time.

v2: rebase on restructured window map.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of limiting the size before we call the mapping
function let the function itself limit the size.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

That should never happen, but make sure that we only warn instead of
crash.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We probably never trigger this, but the logic inside the check is
inverted.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The eDP link rate reported by the DP_MAX_LINK_RATE dpcd register (0xa) is
contradictory to the highest rate supported reported by
EDID (0xc = LINK_RATE_RBR2). The effects of this compounded with commit
'4a8ca46b ("drm/amd/display: Default max bpc to 16 for eDP")' results
in no display modes being found and a dark panel.

For now, simply force the maximum supported link rate for the eDP attached
2018 15" Apple Retina panels.

Additionally, we must also check the firmware revision since the device ID
reported by the DPCD is identical to that of the more capable 16,1,
incorrectly quirking it. We also use said firmware check to quirk the
refreshed 15,1 models with Vega graphics as they use a slightly newer
firmware version.
Tested-by: NAun-Ali Zaidi <admin@kodeit.net>
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Signed-off-by: NAun-Ali Zaidi <admin@kodeit.net>
Signed-off-by: NAditya Garg <gargaditya08@live.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Eliminate the follow smatch warning:
drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c:2246
dp_perform_8b_10b_link_training() warn: inconsistent indenting
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Reported-by: NAbaci Robot <abaci@linux.alibaba.com>
Signed-off-by: NYang Li <yang.lee@linux.alibaba.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

DP2 sequence is triggered only if VESA certified cable is detected.

Force DP2 sequence with uncertified cable for testing purpose.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NWenjing Liu <Wenjing.Liu@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NFangzhi Zuo <Jerry.Zuo@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This version brings along following fixes:
- DC refactor and bug fixes for DP links
- Bug fixes for DP2
- Fix regressions causing display not light up
- Improved debug trace
- Improved DP AUX transfer
- Updated watermark latencies to fix underflows in some modes
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NAric Cyr <aric.cyr@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

 - Correct number of reserved bits in cmd_lock_hw
 - Extend bits of hw_lock_client to allow for more clients
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NAnthony Koo <Anthony.Koo@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Move link_hwss to its own folder as part of DC LIB and break it down
to separate file one for each type of backend for code isolation.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Isolate the way to obtain link_hwss from the actual implemenation of
link_hwss. So the caller can call link_hwss without knowing the
implementation detail of link_hwss.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Clean up dc_link_hwss file in the preparation of breaking it down to
file for each encoder type. We temporarly move the original dp link
functions in link_hwss back to dc_link_dp. We will break dc_link_dp down
after link_hwss is in good shape.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Factor set dp lane settings to link_hwss.

v2: fix statement with no effect warning (Alex)
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Factor set dp link test pattern to link_hwss.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Factor enable/disable dp link output to link hwss.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Acked-by: NWayne Lin <wayne.lin@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[how]
1. move decide det link training link resource before each link training.
2. move disable link for handling vbios case into set all streams
dpms off for link sequence.
3. extract usbc hotplug workaround into its own wa function.
4. Minor syntax changes to improve code readability.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Acked-by: NWayne Lin <wayne.lin@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Factor setup/reset stream encoder to link hwss.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Acked-by: NWayne Lin <wayne.lin@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
This change causes regression, that prevents some systems
from lighting up internal displays.

[How]
Revert this patch until a new solution is ready.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NCharlene Liu <Charlene.Liu@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NZhan Liu <Zhan.Liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
We find some of the driver sequence debug trace for infoframe
update is missing so add it.

[How]
Add the missing sequence debug trace for infoframe update.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NMartin Leung <Martin.Leung@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NLeo (Hanghong) Ma <hanghong.ma@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
The original latencies were causing underflow in some modes.
Resolution: 2880x1620@60p when HDR enable

[How]
1. Replace with the up-to-date watermark values based on new measurments
2. Correct the ddr_wm_table name to DDR5 on DCN31
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NAric Cyr <Aric.Cyr@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NPaul Hsieh <paul.hsieh@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why + How]
Payload reply is unknown and not handled in switch statement.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NAnthony Koo <Anthony.Koo@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NWyatt Wood <wyatt.wood@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>