This option is no longer needed.  The default code paths
are now the only option.

v2: Add HPAGE support and a default for non contiguous maps
v3: Misread 512 pages as MiB ...
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Split late_init into two functions, one (do_late_init) which
just does the hw init, and late_init which calls do_late_init
and schedules the IB test work.  Call do_late_init in
the GPU reset code to run the init code, but not schedule
the IB test code.  The IB test code is called directly
in the gpu reset code so no need to run the IB tests
in a separate work thread.  If we do, we end up racing.

v2: Rework late_init.  Pull out the mgpu fan boost and xgmi
pstate code into late_init so they get called in all cases.
rename the late_init worker thread to delayed work since it's
just the IB tests now which can happen later.  Schedule the
work at init and resume time.  It's not needed at reset time
because the IB tests are called directly.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Cc: Xinhui Pan <xinhui.pan@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add a file that provides a Unique ID for the GPU.
This will persist across machines and is guaranteed to be unique.
This is only available for GFX9 and newer, so older ASICs will not
have this file in the sysfs pool

v2: Store it in adev for ASICs that don't have a hwmgr
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NKent Russell <kent.russell@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v6: Squash in warning fix (Colin Ian King)
v5: Fix warnings (Alex)
v4: fixed mixed delaration and code warnings and minor errors
v3: exposing df funcs in amdgpu_df_funcs in amdgpu.h
v2: moving permonctl/perfmonctr from default to offset

- adding df perfmonctl and perfmonctr registers for df counters
- adding df funcs to set perfmonctl and get perfmonctr for
df and xgmi counters
- exposing df funcs in amdgpu_df_funcs
Signed-off-by: NJonathan Kim <jonathan.kim@amd.com>
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add a sysfs file for reporting the number of PCIe replays (NAKs). This
returns the sum of NAKs received and NAKs generated
Signed-off-by: NKent Russell <kent.russell@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Every ring type can have its own timeout setting.

 - V2: update lockup_timeout parameter format and cosmetic fixes
 - V3: invalidate 0 and negative values
 - V4: update lockup_timeout parameter format
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Remap HDP_MEM_COHERENCY_FLUSH_CNTL and HDP_REG_COHERENCY_FLUSH_CNTL
to an empty page in mmio space. We will later map this page to process
space so application can flush hdp. This can't be done properly at
those registers' original location because it will expose more than
desired registers to process space.

v2: Use explicit register hole location
v3: Moved remapped hdp registers into adev struct
v4: Use more generic name for remapped page
    Expose register offset in kfd_ioctl.h
v5: Move hdp register remap function to nbio ip function
v6: Fixed operator precedence issue and other bugs
Signed-off-by: NOak Zeng <Oak.Zeng@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

syncobj wait/signal operation is appending in command submission.
v2: separate to two kinds in/out_deps functions
v3: fix checking for timeline syncobj
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Cc: Tobias Hector <Tobias.Hector@amd.com>
Cc: Jason Ekstrand <jason@jlekstrand.net>
Cc: Dave Airlie <airlied@redhat.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: NLionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

In VRR mode, proper vblank/pageflip timestamps can only be computed
after the display scanout position has left front-porch. Therefore
delay calls to drm_crtc_handle_vblank(), and thereby calls to
drm_update_vblank_count() and pageflip event delivery, to after the
end of front-porch when in VRR mode.

We add a new vupdate irq, which triggers at the end of the vupdate
interval, ie. at the end of vblank, and calls the core vblank handler
function. The new irq handler is not executed in standard non-VRR
mode, so vblank handling for fixed refresh rate mode is identical
to the past implementation.

v2: Implement feedback by Nicholas and Paul Menzel.
Signed-off-by: NMario Kleiner <mario.kleiner.de@gmail.com>
Acked-by: NHarry Wentland <harry.wentland@amd.com>
Reviewed-by: NNicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This reverts commit 9b638f97.

Adding this to the mapping is complete nonsense and the whole
implementation looks racy. This patch wasn't thoughtfully reviewed
and should be reverted for now.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NLiu, Shaoyun <Shaoyun.Liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Allow RAS feature enable/disable via boot parameter.
Signed-off-by: Nxinhui pan <xinhui.pan@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Applied vdci flush workaround for Vega20 BACO.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Driver vote low to high pstate switch whenever there is an outstanding
XGMI mapping request. Driver vote high to low pstate when all the
outstanding XGMI mapping is terminated.
Signed-off-by: Nshaoyunl <shaoyun.liu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Move pp_feature from the struct of amd_powerplay to amdgpu_device.
Add pp_feature limit for overdrive interface.

v2: put pp_feature into struct amdgpu_pm.
v3: merge feature_mask with pp_feature.
Signed-off-by: NLikun Gao <Likun.Gao@amd.com>
Reviewed-by: NKevin Wang <kevin1.wang@amd.com>
Suggested-by: NAlex Deucher <alexander.deucher@amd.com>
Suggested-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This patch add amdgpu_smu ip block skeleton as placeholder.
We plan use new smu ip block from Vega20.
Signed-off-by: NHuang Rui <ray.huang@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Otherwise we interpret the file private data as drm & amdgpu data
while it might not be, possibly allowing one to get memory corruption.
Signed-off-by: NBas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Different ASIC has different SDMA queue number so
different SDMA doorbell range. Introduce an extra
parameter to sdma_doorbell_range function and set
sdma doorbell range correctly.
Signed-off-by: NOak Zeng <Oak.Zeng@amd.com>
Reviewed-by: NPhilip Yang <Philip.Yang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Used to determine if we need to reset the asic on init due
to the driver having been previously loaded or not shutdown
cleanly.  E.g., kexec or VM passthrough.

v2: rebase
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add a sysfs file that reports the number of bytes transmitted and
received in the last second. This can be used to approximate the PCIe
bandwidth usage over the last second.

v2: Clarify use of mps as estimation of bandwidth
v3: Don't make the file on APUs
v4: Early exit for APUs in the read function, change output to
    display "packets-received packets-sent mps"
v5: fix missing header for si (Alex)
Signed-off-by: NKent Russell <kent.russell@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

driver need to reserve resource for each ctx for
some hw features. so add this limitation.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We don't want KFD processes evicting each other over VRAM usage.
Therefore prevent overcommitting VRAM among KFD applications with
a per-GPU limit. Also leave enough room for page tables on top
of the application memory usage.
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NEric Huang <JinHuiEric.Huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

KIQ in VF’s init delayed by another VF’s reset,
which would cause late_init failed occasionally.
MAX_KIQ_REG_TRY enlarged from 20 to 80 would fix this issue.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NWentao Lou <Wentao.Lou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Use per hive wq to concurrently send reset commands to all nodes
in the hive.

v2:
Switch to system_highpri_wq after dropping dedicated queue.
Fix non XGMI code path KASAN error.
Stop  the hive reset for each node loop if there
is a reset failure on any of the nodes.
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For XGMI hive case do reset in steps where each step iterates over
all devs in hive. This especially important for asic reset
since all PSP FW in hive must come up within a limited time
(around 1 sec) to properply negotiate the link.
Do this by  refactoring  amdgpu_device_gpu_recover and amdgpu_device_reset
into pre_asic_reset, asic_reset and post_asic_reset functions where is part
is exectued for all the GPUs in the hive before going to the next step.

v2: Update names for amdgpu_device_lock/unlock functions.

v3: Introduce per hive locking to avoid multiple resets for GPUs
    in same hive.
v4:
Remove delayed_workqueue()/ttm_bo_unlock_delayed_workqueue() - they
are copy & pasted over from radeon and on amdgpu there isn't
any reason for that any more.
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is prep work for updating each PSP FW in hive after
GPU reset.
Split into build topology SW state and update each PSP FW in the hive.
Save topology and count of XGMI devices for reuse.

v2: Create seperate header for XGMI.
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Move doorbell structures, enum definitions and helper functions
from amdgpu.h to amdgpu_doorbell.h. No functional change
Signed-off-by: NOak Zeng <ozeng@amd.com>
Proposed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This introduces new doorbell layout for vega20 and future asics

v2: Use enum definition instead of hardcoded value
Signed-off-by: NOak Zeng <ozeng@amd.com>
Suggested-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Suggested-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: Use enum definition instead of hardcoded value
v3: Remove unused enum definition
Signed-off-by: NOak Zeng <ozeng@amd.com>
Suggested-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Suggested-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is preparation to move doorbell index initialization
to amdgpu_asic_funcs
Signed-off-by: NOak Zeng <ozeng@amd.com>
Proposed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is a preparation work to make reserved doorbell index per device,
instead of using a global macro definition. By doing this, we can easily
change doorbell layout for future ASICs while not affecting ASICs in
production.
Signed-off-by: NOak Zeng <ozeng@amd.com>
Suggested-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Suggested-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

ACPI ATIF has a function called query
backlight transfer characteristics.  Among the
information returned by this function is
the minimum and maximum input signals for the
backlight

Call that function on ACPI init.  When DM
backlight device is updated, copy over the
backlight caps into DM, but only once.  Use
the backlight caps in the backlight-to-dc
calculation
Signed-off-by: NDavid Francis <David.Francis@amd.com>
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Similar to ppfeaturemask.  Allows you to selectively enable/disable
DC features.
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Similar to ppfeaturemask.  Allows you to selectively enable/disable
DC features.
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

In baremetal, also need to reserve csa for preemption.
so move the csa related code out of sriov.
Reviewed-by: NMonk Liu <Monk.Liu@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signal a fault to the scheduler on an illegal instruction or register
access violation instead of kicking of the reset handler directly.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NFrank.Min <Frank.Min@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Move in_suspend flag to adev from gfx, so
can be used in other ip blocks, also keep
consistent with gpu_in_reset flag.
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

the struct was not in use any more.
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Count all GPU instances from AMD(including iGPUs and
dGPUs) in the system.

V2: drop unnecessary initialization for other gpu_info
    members except mutex
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Correct the format

For vega10 sriov, the sdma doorbell must be fixed as follow to keep the
same setting with host driver, or it will happen conflicts.
Signed-off-by: NEmily Deng <Emily.Deng@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>