This adds support to discover the  buffer objects that belong to a
process being checkpointed. The data corresponding to these buffer
objects is returned to user space plugin running under criu master
context which then stores this info to recreate these buffer objects
during a restore operation.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This IOCTL op is expected to be called as a precursor to the actual
Checkpoint operation. This does the basic discovery into the target
process seized by CRIU and relays the information to the userspace that
utilizes it to start the Checkpoint operation via another dedicated
IOCTL op.

The process_info IOCTL op determines the number of GPUs, buffer objects
that are associated with the target process, its process id in
caller's namespace since /proc/pid/mem interface maybe used to drain
the contents of the discovered buffer objects in userspace and getpid
returns the pid of CRIU dumper process. Also the pid of a process
inside a container might be different than its global pid so return
the ns pid.
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Checkpoint-Restore in userspace (CRIU) is a powerful tool that can
snapshot a running process and later restore it on same or a remote
machine but expects the processes that have a device file (e.g. GPU)
associated with them, provide necessary driver support to assist CRIU
and its extensible plugin interface. Thus, In order to support the
Checkpoint-Restore of any ROCm process, the AMD Radeon Open Compute
Kernel driver, needs to provide a set of new APIs that provide
necessary VRAM metadata and its contents to a userspace component
(CRIU plugin) that can store it in form of image files.

This introduces some new ioctls which will be used to checkpoint-Restore
any KFD bound user process. KFD only allows ioctl calls from the same
process that opened the KFD file descriptor. Since these ioctls are
expected to be called from a KFD criu plugin which has elevated ptrace
attached privileges and CAP_CHECKPOINT_RESTORE capabilities attached with
the file descriptors so modify KFD to allow such calls.

(API redesigned by David Yat Sin)
Suggested-by: NFelix Kuehling <felix.kuehling@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NDavid Yat Sin <david.yatsin@amd.com>
Signed-off-by: NRajneesh Bhardwaj <rajneesh.bhardwaj@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

MESA polls for errors every 2-3 seconds. Printing with dev_info() causes
the dmesg log to fill up with the same message, e.g,

[18028.206676] amdgpu 0000:0b:00.0: amdgpu: df doesn't config ras function.

Make it dev_dbg_once(), as it isn't something correctible during boot or
thereafter, so printing just once is sufficient. Also sanitize the message.

Cc: Alex Deucher <Alexander.Deucher@amd.com>
Cc: Hawking Zhang <Hawking.Zhang@amd.com>
Cc: John Clements <john.clements@amd.com>
Cc: Tao Zhou <tao.zhou1@amd.com>
Cc: yipechai <YiPeng.Chai@amd.com>
Fixes: 8b0fb0e9 ("drm/amdgpu: Modify gfx block to fit for the unified ras block data and ops")
Signed-off-by: NLuben Tuikov <luben.tuikov@amd.com>
Reviewed-by: NAlex Deucher <Alexander.Deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Some people complained about the name and this matches much
more Linux naming conventions for object functions.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Whenever a bo_va structure is added or removed the VM and eventually
added BO should be locked.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Assigning 0L to a pointer variable caused the following warning:

drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dsc/rc_calc_fpu.c:71:40:
warning: Using plain integer as NULL pointer

In order to remove this warning, this commit assigns a NULL pointer to
the pointer variable that caused this issue.
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NMagali Lemes <magalilemes00@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

   (v3)
     Rewrote patchset to order patches as (API, hw impl, usecase)

     - added API for new power management function emit_clk_levels
       This function should duplicate the functionality of print_clk_levels,
       but this solution passes the buffer base and write offset down the stack.
     - new powerplay function emit_clock_levels, implemented by smu_emit_ppclk_levels()
       This function parallels the implementation of smu_print_ppclk_levels and
       calls emit_clk_levels, and allows the returns of errors
     - new helper function smu_convert_to_smuclk called by smu_print_ppclk_levels and
       smu_emit_ppclk_levels
Signed-off-by: NDarren Powell <darren.powell@amd.com>
Reviewed-By: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

trace_amdgpu_vm_update_ptes trace unable to log when nptes too large
Signed-off-by: NSomalapuram Amaranath <Amaranath.Somalapuram@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

dGPUs connected to Intel systems configured for suspend to idle
will not have the power rails cut at suspend and resetting the GPU
may lead to problematic behaviors.

Fixes: e25443d2 ("drm/amdgpu: add a dev_pm_ops prepare callback (v2)")
Link: https://gitlab.freedesktop.org/drm/amd/-/issues/1879Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NMario Limonciello <mario.limonciello@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We ran into the problem that clearing really larger buffer (60GiB) caused an
SDMA timeout.

Restructure the function to use the dst window instead of mapping the whole
buffer into the GART and then fill only 2MiB/256MiB chunks at a time.

v2: rebase on restructured window map.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of limiting the size before we call the mapping
function let the function itself limit the size.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

That should never happen, but make sure that we only warn instead of
crash.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We probably never trigger this, but the logic inside the check is
inverted.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The eDP link rate reported by the DP_MAX_LINK_RATE dpcd register (0xa) is
contradictory to the highest rate supported reported by
EDID (0xc = LINK_RATE_RBR2). The effects of this compounded with commit
'4a8ca46b ("drm/amd/display: Default max bpc to 16 for eDP")' results
in no display modes being found and a dark panel.

For now, simply force the maximum supported link rate for the eDP attached
2018 15" Apple Retina panels.

Additionally, we must also check the firmware revision since the device ID
reported by the DPCD is identical to that of the more capable 16,1,
incorrectly quirking it. We also use said firmware check to quirk the
refreshed 15,1 models with Vega graphics as they use a slightly newer
firmware version.
Tested-by: NAun-Ali Zaidi <admin@kodeit.net>
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Signed-off-by: NAun-Ali Zaidi <admin@kodeit.net>
Signed-off-by: NAditya Garg <gargaditya08@live.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Eliminate the follow smatch warning:
drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c:2246
dp_perform_8b_10b_link_training() warn: inconsistent indenting
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Reported-by: NAbaci Robot <abaci@linux.alibaba.com>
Signed-off-by: NYang Li <yang.lee@linux.alibaba.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

DP2 sequence is triggered only if VESA certified cable is detected.

Force DP2 sequence with uncertified cable for testing purpose.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NWenjing Liu <Wenjing.Liu@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NFangzhi Zuo <Jerry.Zuo@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This version brings along following fixes:
- DC refactor and bug fixes for DP links
- Bug fixes for DP2
- Fix regressions causing display not light up
- Improved debug trace
- Improved DP AUX transfer
- Updated watermark latencies to fix underflows in some modes
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NAric Cyr <aric.cyr@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

 - Correct number of reserved bits in cmd_lock_hw
 - Extend bits of hw_lock_client to allow for more clients
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NAnthony Koo <Anthony.Koo@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Move link_hwss to its own folder as part of DC LIB and break it down
to separate file one for each type of backend for code isolation.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Isolate the way to obtain link_hwss from the actual implemenation of
link_hwss. So the caller can call link_hwss without knowing the
implementation detail of link_hwss.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Clean up dc_link_hwss file in the preparation of breaking it down to
file for each encoder type. We temporarly move the original dp link
functions in link_hwss back to dc_link_dp. We will break dc_link_dp down
after link_hwss is in good shape.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Factor set dp lane settings to link_hwss.

v2: fix statement with no effect warning (Alex)
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Factor set dp link test pattern to link_hwss.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NJun Lei <Jun.Lei@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Factor enable/disable dp link output to link hwss.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Acked-by: NWayne Lin <wayne.lin@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[how]
1. move decide det link training link resource before each link training.
2. move disable link for handling vbios case into set all streams
dpms off for link sequence.
3. extract usbc hotplug workaround into its own wa function.
4. Minor syntax changes to improve code readability.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Acked-by: NWayne Lin <wayne.lin@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[why]
Factor setup/reset stream encoder to link hwss.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Acked-by: NWayne Lin <wayne.lin@amd.com>
Signed-off-by: NWenjing Liu <wenjing.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
This change causes regression, that prevents some systems
from lighting up internal displays.

[How]
Revert this patch until a new solution is ready.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NCharlene Liu <Charlene.Liu@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NZhan Liu <Zhan.Liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
We find some of the driver sequence debug trace for infoframe
update is missing so add it.

[How]
Add the missing sequence debug trace for infoframe update.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NMartin Leung <Martin.Leung@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NLeo (Hanghong) Ma <hanghong.ma@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
The original latencies were causing underflow in some modes.
Resolution: 2880x1620@60p when HDR enable

[How]
1. Replace with the up-to-date watermark values based on new measurments
2. Correct the ddr_wm_table name to DDR5 on DCN31
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NAric Cyr <Aric.Cyr@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NPaul Hsieh <paul.hsieh@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why + How]
Payload reply is unknown and not handled in switch statement.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NAnthony Koo <Anthony.Koo@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NWyatt Wood <wyatt.wood@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
Certain configurations will result in link encoder
to not be assigned to the link at the time we apply
cable ID logic. We should skip it in those cases.

[How]
Check if link_enc is not null before applying
cable ID.
Tested-by: NDaniel Wheeler <daniel.wheeler@amd.com>
Reviewed-by: NWenjing Liu <Wenjing.Liu@amd.com>
Acked-by: NStylon Wang <stylon.wang@amd.com>
Signed-off-by: NGeorge Shen <george.shen@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

A lot of below message are outputed in SRIOV case.
amdgpu: indirect registers access through rlcg is not supported

Also drop redundant ret set, as it's initialized to be false already.

Fixes: 29dbcac8 ("drm/amdgpu: add helper to query rlcg reg access flag")
Signed-off-by: NGuchun Chen <guchun.chen@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Fix uninitialized variable use
warning: variable 'reg_access_ctrl' is uninitialized when used here [-Wuninitialized]
     scratch_reg0 = (void __iomem *)adev->rmmio + 4 * reg_access_ctrl->scratch_reg0;

Fixes: 5d447e29 ("drm/amdgpu: add helper for rlcg indirect reg access")
Reported-by: Nkernel test robot <yujie.liu@intel.com>
Signed-off-by: NLijo Lazar <lijo.lazar@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NGuchun Chen <guchun.chen@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

1. The infinite loop causing soft lock occurs on multiple amdgpu cards
   supporting ras feature.
2. This a workaround patch to fix 6492e1b0.
   It is valid for multiple amdgpu cards of the same type.
3. The root cause is that each GPU card device has a separate .ras_list
   link header, but the instance and linked list node of each ras block
   are unique. When each device is initialized, each ras instance will
   repeatedly add link node to the device every time. In this way, only
   the .ras_list of the last initialized device is completely correct.
   the .ras_list->prev and .ras_list->next of the device initialzied
   before can still point to the correct ras instance, but the prev
   pointer and next pointer of the pointed ras instance both point to
   the last initialized device's .ras_ list instead of the beginning
   .ras_ list. When using list_for_each_entry_safe searches for
   non-existent Ras nodes on devices other than the last device, the
   last ras instance next pointer cannot always be equal to the
   beginning .ras_list, so that the loop cannot be terminated, the
   program enters a infinite loop.
 BTW: Since the data and initialization process of each card are the same,
      the link list between ras instances will not be destroyed every time
      the device is initialized.
 4. The soft locked logs are as follows:
[  262.165690] CPU: 93 PID: 758 Comm: kworker/93:1 Tainted: G           OE     5.13.0-27-generic #29~20.04.1-Ubuntu
[  262.165695] Hardware name: Supermicro AS -4124GS-TNR/H12DSG-O-CPU, BIOS T20200717143848 07/17/2020
[  262.165698] Workqueue: events amdgpu_ras_do_recovery [amdgpu]
[  262.165980] RIP: 0010:amdgpu_ras_get_ras_block+0x86/0xd0 [amdgpu]
[  262.166239] Code: 68 d8 4c 8d 71 d8 48 39 c3 74 54 49 8b 45 38 48 85 c0 74 32 44 89 fa 44 89 e6 4c 89 ef e8 82 e4 9b dc 85 c0 74 3c 49 8b 46 28 <49> 8d 56 28 4d 89 f5 48 83 e8 28 48 39 d3 74 25 49 89 c6 49 8b 45
[  262.166243] RSP: 0018:ffffac908fa87d80 EFLAGS: 00000202
[  262.166247] RAX: ffffffffc1394248 RBX: ffff91e4ab8d6e20 RCX: ffffffffc1394248
[  262.166249] RDX: ffff91e4aa356e20 RSI: 000000000000000e RDI: ffff91e4ab8c0000
[  262.166252] RBP: ffffac908fa87da8 R08: 0000000000000007 R09: 0000000000000001
[  262.166254] R10: ffff91e4930b64ec R11: 0000000000000000 R12: 000000000000000e
[  262.166256] R13: ffff91e4aa356df8 R14: ffffffffc1394320 R15: 0000000000000003
[  262.166258] FS:  0000000000000000(0000) GS:ffff92238fb40000(0000) knlGS:0000000000000000
[  262.166261] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  262.166264] CR2: 00000001004865d0 CR3: 000000406d796000 CR4: 0000000000350ee0
[  262.166267] Call Trace:
[  262.166272]  amdgpu_ras_do_recovery+0x130/0x290 [amdgpu]
[  262.166529]  ? psi_task_switch+0xd2/0x250
[  262.166537]  ? __switch_to+0x11d/0x460
[  262.166542]  ? __switch_to_asm+0x36/0x70
[  262.166549]  process_one_work+0x220/0x3c0
[  262.166556]  worker_thread+0x4d/0x3f0
[  262.166560]  ? process_one_work+0x3c0/0x3c0
[  262.166563]  kthread+0x12b/0x150
[  262.166568]  ? set_kthread_struct+0x40/0x40
[  262.166571]  ret_from_fork+0x22/0x30

Fixes: 6492e1b0 ("drm/amdgpu: Unify ras block interface for each ras block")
Signed-off-by: Nyipechai <YiPeng.Chai@amd.com>
Reviewed-by: NJohn Clements <john.clements@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

[Why]
There is underflow / visual corruption DCN301, for high
bandwidth MST DSC configurations such as 2x1440p144 or 2x4k60.

[How]
Use up-to-date watermark values for DCN301.
Reviewed-by: NZhan Liu <zhan.liu@amd.com>
Signed-off-by: NAgustin Gutierrez <agustin.gutierrez@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

All warnings (new ones prefixed by >>):

   drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_svm.c: In function
'svm_range_deferred_list_work':
>> drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_svm.c:2067:22: warning:
variable 'p' set but not used [-Wunused-but-set-variable]
    2067 |  struct kfd_process *p;
         |

Fixes: 367c9b0f ("drm/amdkfd: Ensure mm remain valid in svm deferred_list work")
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NPhilip Yang <Philip.Yang@amd.com>
Reviewed-By: NHarish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

There are quite a few spelling mistakes in various function names
and error messages. Fix these.
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Signed-off-by: NColin Ian King <colin.i.king@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

'amdgpu_dpm.h' included in 'arcturus_ppt.c' is duplicated.
Reported-by: NZeal Robot <zealci@zte.com.cn>
Signed-off-by: NChangcheng Deng <deng.changcheng@zte.com.cn>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

'linux/pci.h' included in 'amdgpu_device.c' is duplicated.
Reported-by: NZeal Robot <zealci@zte.com.cn>
Signed-off-by: NChangcheng Deng <deng.changcheng@zte.com.cn>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>