Drop the "kernel" and sg parameter and give the BO type to create
explicit to amdgpu_bo_create instead of figuring it out from the
parameters.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NRoger He <Hongbo.He@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Just set the GTT domain as mandatory, so that the BO is validated into
it on first use. This allows us to setup the sg table later on.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NRoger He <Hongbo.He@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of moving this to the SYSTEM domain just drop the backing store
and let the resulting allocation be freed.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NRoger He <Hongbo.He@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of calculating the size in bytes just to recalculate the number
of pages from it pass the BO directly to the function.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NRoger He <Hongbo.He@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

this patch actually refactor mailbox implmentations, and
all below changes are needed together to fix all those mailbox
handshake issues exposured by heavey TDR test.

1)refactor all mailbox functions based on byte accessing for mb_control
reason is to avoid touching non-related bits when writing trn/rcv part of
mailbox_control, this way some incorrect INTR sent to hypervisor
side could be avoided, and it fixes couple handshake bug.

2)trans_msg function re-impled: put a invalid
logic before transmitting message to make sure the ACK bit is in
a clear status, otherwise there is chance that ACK asserted already
before transmitting message and lead to fake ACK polling.
(hypervisor side have some tricks to workaround ACK bit being corrupted
by VF FLR which hase an side effects that may make guest side ACK bit
asserted wrongly), and clear TRANS_MSG words after message transferred.

3)for mailbox_flr_work, it is also re-worked: it takes the mutex lock
first if invoked, to block gpu recover's participate too early while
hypervisor side is doing VF FLR. (hypervisor sends FLR_NOTIFY to guest
before doing VF FLR and sentds FLR_COMPLETE after VF FLR done, and
the FLR_NOTIFY will trigger interrupt to guest which lead to
mailbox_flr_work being invoked)

This can avoid the issue that mailbox trans msg being cleared by its VF FLR.

4)for mailbox_rcv_irq IRQ routine, it should only peek msg and schedule
mailbox_flr_work, instead of ACK to hypervisor itself, because FLR_NOTIFY
msg sent from hypervisor side doesn't need VF's ACK (this is because
VF's ACK would lead to hypervisor clear its trans_valid/msg, and this
would cause handshake bug if trans_valid/msg is cleared not due to
correct VF ACK but from a wrong VF ACK like this "FLR_NOTIFY" one)

This fixed handshake bug that sometimes GUEST always couldn't receive
"READY_TO_ACCESS_GPU" msg from hypervisor.

5)seperate polling time limite accordingly:
POLL ACK cost no more than 500ms
POLL MSG cost no more than 12000ms
POLL FLR finish cost no more than 500ms

6) we still need to set adev into in_gpu_reset mode after we received
FLR_NOTIFY from host side, this can prevent innocent app wrongly succesed
to open amdgpu dri device.

FLR_NOFITY is received due to an IDLE hang detected from hypervisor side
which indicating GPU is already die in this VF.

v2:
use MACRO as the offset of mailbox_control register
don't test if NOTIFY_CMPL event in rcv_msg since it won't
recieve that message anymore
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NPixel Ding <Pixel.Ding@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

mailbox registers can be accessed with a byte boundry according
to BIF team, so this patch prepares register byte access
and will be used by following patches.

Actually, for mailbox registers once the byte field is touched even not changed,
the mailbox behaves, so we need the byte width accessing to those sort of regs.
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NPixel Ding <Pixel.Ding@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The vram type for dGPU is stored in umc_info while sys mem type
for APU is stored in integratedsysteminfo
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The commit d296278fd372003fc69588acfd0c0c5edbdf4874 added support for
detecting DDR4 but omitted the label that is printed out in
amdgpu_bo_init() resulting in a KASAN error.
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The amdgpu_ucode_fini_bo should be called after gfx_v8_0_hw_fini,
or it will have KCQ disable failed issue.

For Tonga, as it firstly finishes SMC block, and the SMC hw fini
will call amdgpu_ucode_fini, which will lead the amdgpu_ucode_fini_bo
called before gfx_v8_0_hw_fini, this is incorrect.
Signed-off-by: NEmily Deng <Emily.Deng@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The amdgpu_pm_sysfs_fini should call before amdgpu_device_ip_fini,
or the adev->pm.dpm_enabled would be set to 0, then the device files
related to pp won't be removed by amdgpu_pm_sysfs_fini when unload
driver.
Signed-off-by: NEmily Deng <Emily.Deng@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We don't need the page array for prime shared BOs, stop allocating it.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NRoger He <Hongbo.He@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When UVD is in VM mode, there is not uvd handle exchanged,
uvd.handles are always 0. So vcpu_bo always need save,
Otherwise amdgpu driver will fail during suspend/resume.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=105021Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NLeo Liu <leo.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Max uvd handles should use adev->uvd.max_handles instead of
AMDGPU_MAX_UVD_HANDLES here.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NLeo Liu <leo.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

This allows access to pages allocated through the driver with optional
IOMMU mapping.

v2: Fix number of bytes copied and add write method
v3: drop check for kmap return
Original-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The sdma wptr polling memory is not fast enough, then the sdma
wptr register will be random, and not equal to sdma rptr, which
will cause sdma engine hang when load driver, so clean up the sdma
wptr directly to fix this issue.

v2:add comment above the code and correct coding style
Reviewed-by: NXiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NEmily Deng <Emily.Deng@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

to catch error that may schedule in atomic context early on
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Disable the workaround on imported BOs as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexdeucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We can use ttm->bdev instead.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexdeucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Allow evicting all BOs from the GTT domain.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexdeucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

delete those cgs interfaces:
amdgpu_cgs_alloc_gpu_mem
amdgpu_cgs_free_gpu_mem
amdgpu_cgs_gmap_gpu_mem
amdgpu_cgs_gunmap_gpu_mem
amdgpu_cgs_kmap_gpu_mem
amdgpu_cgs_kunmap_gpu_mem
Reviewed-by: NAlex Deucher <alexdeucher@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Reviewed-by: NAlex Deucher <alexdeucher@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add power profiling mode dynamic switch based on the workloads.
Currently, support Cumpute, VR, Video, 3D,power saving with Cumpute
have highest prority, power saving have lowest prority.

in manual dpm mode, driver will stop auto switch, just save the client's
requests. user can set power profiling mode through sysfs.

when exit manual dpm mode, driver will response the client's requests.
switch based on the client's prority.

v2: squash in fixes from Rex
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The gfx/compute profiling mode switch is only for internally
test. Not a complete solution and unexpectly upstream.
so revert it.
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Always set the graphics values to the max for the
asic type.  E.g., some 1 RB chips are actually 1 RB chips,
others are actually harvested 2 RB chips.

Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=99353Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Get gpu info through adev directly in powerplay
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

it is required if a platform supports PCIe root complex
core voltage reduction. After receiving this notification,
SBIOS can apply default PCIe root complex power policy.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

use adev as input parameter to create powerplay instance
directly. delete cgs wrap layer for power play create.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For consistency with other DCE generations.

HPD IRQs appear to be working fine.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The ring status can change during GPU reset, but we still need to be
able to schedule TTM buffer moves in the meantime.

Otherwise we can ran into problems because of aborted move/fill
operations during GPU resets.

v2: still check if ring is available during direct submit.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NChunming zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When we reset the GPU we also disable/enable the SDMA, but we don't want
to change TTM idea of the VRAM size in the middle of that.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NChunming zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of setting the active VRAM size directly provide a the info if
we can use the buffer functions or not.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NChunming zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Those belong to the TTM handling.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Rather than querying it every time we need it.
Also fixes a crash in VM pass through if there is no
root bridge because the cached value fetch already checks
this properly.

v2: fix includes

Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=105244Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Rex Zhu<rezhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

The read/write pointers on sdma4 devices increment
beyond the ring size and should be masked.  Tested
on my Ryzen 2400G.
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

sometimes GPU is switched to other VFs and won't swich
back soon, so the kiq reg access will not signal within
a short period, instead of busy waiting a long time(MAX_KEQ_REG_WAIT)
and returning TMO we can istead sleep 5ms and try again
later (non irq context)

And since the waiting in kiq_r/weg is busy wait, so MAX_KIQ_REG_WAIT
shouldn't set to a long time, set it to 10ms is more appropriate.

if gpu already in reset state, don't retry the KIQ reg access
otherwise it would always hang because KIQ was already die usually.

v2:
replace schedule() with msleep() for the wait

v3:
use while loop for the wait repeating
use macros for the sleep period
more description for it

v4:
drop unused variable
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com
Reviewed-by: NPixel Ding <Pixel.Ding@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

1)create a routine "handle_vram_lost" to do the vram
recovery, and put it into amdgpu_device_reset/reset_sriov,
this way no need of the extra paramter to hold the
VRAM LOST information and the related macros can be removed.

3)show vram_recover failure if time out, and set TMO equal to
lockup_timeout if vram_recover is under SRIOV runtime mode.

4)report error if any ip reset failed for SR-IOV
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

found recover_vram_from_shadow sometimes get executed
in paralle with SDMA scheduler, should stop all
schedulers before doing gpu reset/recover
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Tested-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Some were missing the close parens around options.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When using CPU to update page table, we need to kmap all the PDs/PTs after
they are allocated and that requires a TLB shot down on each CPU, which is
quite heavy.

Instead, we map the whole visible VRAM to a kernel address at once. Pages
can be obtained from the offset.

v2: move the mapping base from gmc to amdgpu_mman structure, and the
    implementation in amdgpu_ttm_* functions
Signed-off-by: NAmber Lin <Amber.Lin@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>