Signed-off-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The hardware can use huge pages to map 2MB of address space with only one PDE.

v2: few cleanups and rebased
v3: skip PT updates if we are using the PDE
v4: rebased, added support for CPU based updates
v5: fix CPU based updates once more
v6: fix ndw estimation
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-and-tested-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The fragment bits work differently for Vega10 compared to previous generations.

Increase the fragment size to 2MB for now to better handle that.

v2: handle the hardware setup as well
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-and-tested-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Currently, get clock info from default clk of pm if dpm is disable.
Buf SRIOV doesn't support dpm and pm, can't get anything from pm.
Only get clock info only from default clk of amdgpu for SRIOV.

And driver get pm default clk also from amdgpu default clk and never
be changed by others. So use amdgpu default clk value for SRIOV
and non-dpm cases.
Signed-off-by: NXiangliang Yu <Xiangliang.Yu@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

SRIOV won't do vbios post in guest OS, and the mmMC_VM_FB_LOCATION
is pf and vf copy, so still need to program fb location for SRIOV.

v2: No need to stop mc, and update gmc_v8_0_vram_gtt_location as well.

v3: New line after the stack variables

BUG: SWDEV-126629
Signed-off-by: NEmily Deng <Emily.Deng@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Now asd firmware is not ready for psp v10, will enable it when it's available
Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

1, for sriov, we need 8dw for the gfx fence due to CP
behaviour
2, cleanup wrong logic in wptr/rptr wb alloc and free

Change-Id: Ifbfed17a4621dae57244942ffac7de1743de0294
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Signed-off-by: NXiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Allows gdb to access contents of user mode mapped VRAM BOs.

v2: return error for non-VRAM pools
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Allows gdb to access contents of user mode mapped BOs. System memory
is handled by TTM using kmap. Other memory pools require a new driver
callback in ttm_bo_driver.

v2:
* kmap only one page at a time
* swap in BO if needed
* make driver callback more generic to handle private memory pools
* document callback return value
* WARN_ON -> WARN_ON_ONCE
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NEric Huang <JinHuiEric.Huang@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: fix the SOS loading failure for PSP v3.1
Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Cc: stable@vger.kernel.org
Acked-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Acked-by: Huang Rui <ray.huang@amd.com> (v1)
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Copy the approach taken by gfx8, which simplifies the code, and set the
instance index properly. The latter is required for debugging, e.g. for
reading wave status by UMR.
Signed-off-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

In RCU read-side critical sections, blocking or sleeping is prohibited.

v2: Unlock RCU for the code path where result==NULL. (David Zhou)
    Update subject

Tested-by and reported by: Dave Airlie <airlied@redhat.com>

[  141.965723] =============================
[  141.965724] WARNING: suspicious RCU usage
[  141.965726] 4.12.0-rc7 #221 Not tainted
[  141.965727] -----------------------------
[  141.965728] /home/airlied/devel/kernel/linux-2.6/include/linux/rcupdate.h:531
Illegal context switch in RCU read-side critical section!
[  141.965730]
               other info that might help us debug this:

[  141.965731]
               rcu_scheduler_active = 2, debug_locks = 0
[  141.965732] 1 lock held by amdgpu_cs:0/1332:
[  141.965733]  #0:  (rcu_read_lock){......}, at: [<ffffffffa01a0d07>]
amdgpu_bo_list_get+0x0/0x109 [amdgpu]
[  141.965774]
               stack backtrace:
[  141.965776] CPU: 6 PID: 1332 Comm: amdgpu_cs:0 Not tainted 4.12.0-rc7 #221
[  141.965777] Hardware name: To be filled by O.E.M. To be filled by
O.E.M./M5A97 R2.0, BIOS 2603 06/26/2015
[  141.965778] Call Trace:
[  141.965782]  dump_stack+0x68/0x92
[  141.965785]  lockdep_rcu_suspicious+0xf7/0x100
[  141.965788]  ___might_sleep+0x56/0x1fc
[  141.965790]  __might_sleep+0x68/0x6f
[  141.965793]  __mutex_lock+0x4e/0x7b5
[  141.965817]  ? amdgpu_bo_list_get+0xa4/0x109 [amdgpu]
[  141.965820]  ? lock_acquire+0x125/0x1b9
[  141.965844]  ? amdgpu_bo_list_set+0x464/0x464 [amdgpu]
[  141.965846]  mutex_lock_nested+0x16/0x18
[  141.965848]  ? mutex_lock_nested+0x16/0x18
[  141.965872]  amdgpu_bo_list_get+0xa4/0x109 [amdgpu]
[  141.965895]  amdgpu_cs_ioctl+0x4a0/0x17dd [amdgpu]
[  141.965898]  ? radix_tree_node_alloc.constprop.11+0x77/0xab
[  141.965916]  drm_ioctl+0x264/0x393 [drm]
[  141.965939]  ? amdgpu_cs_find_mapping+0x83/0x83 [amdgpu]
[  141.965942]  ? trace_hardirqs_on_caller+0x16a/0x186
Signed-off-by: NAlex Xie <AlexBin.Xie@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The test was relaxed a bit to much.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-and-Tested-by: NRoger He <Hongbo.He@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

No need to try to map them every time.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When a BO is moved or destroyed it shouldn't be kmapped any more.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

No need to do this after every single update.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Handy for debugging.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Printing a warning into the logs that we will certainly run into a BUG() is
completely nonsense, the BUG() is more than noisy enough.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We need to wait with the correct owner on unmap operations or otherwise can run
into VM faults.

Also always wait for the page directory since this is where the reservation
object comes from. So rename the function to amdgpu_vm_wait_pd instead as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We don't have any update fence in that case, so the need
for flushing isn't detected automatically.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Spreading them causes performance regressions using compute
queues on Polaris 11.

Cc: Jim Qu <jim.qu@amd.com>
Acked-by: NJim Qu <Jim.Qu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NJim Qu <Jim.Qu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is just future proofing code, not something that can be triggered
in real life.  We're testing to make sure we don't shift wrap when we
do "1ull << i" so "i" has to be in the 0-63 range.  If it's 64 then we
have gone too far.
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This allows us to read the vbios image directly from ROM.
This is already implemented for other asics, but was not
yet available for SI.
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Maybe a leftover from bringup?
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Rather than the legacy atombios version.
Acked-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The information has moved to different tables, notably
smu_info for core refclk and umc_info for mem refclk.
Acked-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

A subset of VM fault types currently send retry XNACK to the client.
This causes a storm of interrupts from the VM to the host.

Until the storm is throttled by other means send no-retry XNACK for
all fault types instead. No change in behavior to the client which
will stall indefinitely with the current configuration in any case.
Improves system stability under GC or MMHUB faults.
Signed-off-by: NJay Cornwall <Jay.Cornwall@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NJohn Bridgman <John.Bridgman@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NYong Zhao <yong.zhao@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Set a configurable SDMA phase quantum when enabling SDMA context
switching. The default value significantly reduces SDMA latency
in page table updates when user-mode SDMA queues have concurrent
activity, compared to the initial HW setting.
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NAndres Rodriguez <andres.rodriguez@amd.com>
Reviewed-by: NShaoyun Liu <shaoyun.liu@amd.com>
Acked-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable SDMA context switching on CIK (copied from sdma_v3_0.c).
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>