Flush and invalidate the HDP caches.

v2: fix typo in comment
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This patch changes pcie_gen_cap magic code to macro to make it more
readable.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NHuang Rui <ray.huang@amd.com>
Cc: Eric Huang <JinHuiEric.Huang@amd.com>
Cc: Ken Wang <Qingqing.Wang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Not used outside of gfx7.
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

gfx8 already uses them.  Remove the direct exports and
use the callbacks fpr gfx7.
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It's gfx IP specific, not asic specific, so move to a
gfx callback.
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It's gfx IP specific, not asic specific, so move to a
gfx callback.
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Only used in the gmc IP modules so just call the local
function directly.
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

These are not used outside of the respective gmc ip modules.
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We don't need to validate them again if the eviction counter didn't changed.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Keep track of the number of evictions since boot.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

irq need to update when gpu reset happens.
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This patch is a port of similar patch for amdgpu
when PP is disabled. Since the code flow is little
different when PP is enabled, we need to make sure
the patch is applied for PP enabled path as well.

With the current code, when we boot with the amdgpu
driver enabled and loaded, the VCE also automatically
remains enabled since bootup. This can be verified from
the output of amdgpu_pm_info. It does not matter whether
we boot into command line directly or into X, the VCE
stays enabled the entire time.

This patch addresses the issue and makes sure that
VCE is turned on only during playback, and remains
disabled otherwise.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NArindam Nath <arindam.nath@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

With the current code, when we boot with the amdgpu
driver enabled and loaded, the VCE also automatically
remains enabled since bootup. This can be verified from
the output of amdgpu_pm_info. It does not matter whether
we boot into command line directly or into X, the VCE
stays enabled the entire time.

This patch addresses the issue and makes sure that
VCE is turned on only during playback, and remains
disaled otherwise.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NArindam Nath <arindam.nath@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

No need to stall the pipe when we are using firmware with the
fix.
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: do not overwrite register when bitmap is zero
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: do not overwrite register when bitmap is zero
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This parameter will allow disabling individual CUs on module load, e.g.
amdgpu.disable_cu=2.0.3,2.0.4 to disable CUs 3 and 4 of SE2.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

On workstation cards with ECC vram, the entirety of vram is cleared to 0
on asic init to set the ECC status correctly.  On non ECC boards, I don't
think they do any explicit clearing, but the vram controller is reset
which may cause issues with the data there.
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

In the AMD powerplay driver, a pointer is checked for validity by
comparing against an integer '0', which causes a harmless warning
when building with "make W=1":

drivers/gpu/drm/amd/amdgpu/../powerplay/hwmgr/processpptables.c:1502:16: error: ordered comparison of pointer with integer zero [-Werror=extra]

This changes the code to the more conventional "if (pointer)" check.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This boosts Xonotic from 38fps to 47fps when artificially limiting VRAM to
256MB for testing. It should improve all CPU bound rendering situations
where we have a lot of swapping to/from VRAM.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When we pipeline evictions the page directory could already be
moving somewhere else when grab_id is called.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Free up the memory immediately, remember the last eviction for each domain and
make new allocations depend on the last eviction to be completed.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

As far as I can see no need for a custom implementation any more.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of using the flag just remember the fence of the last move operation.

This avoids waiting for command submissions pipelined after the move, but
before accessing the BO with the CPU again.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It isn't used and not waiting for the GPU after scheduling a move is
actually quite dangerous.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Not needed any more.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Otherwise we could update the VM page tables while the move is only scheduled.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Final part to avoid pre move waits.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

That is unnecessary now.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

That is unnecessary now.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When we want to pipeline accelerated moves we need to wait in the fallback path.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Wait for idle before moving the BO in all drivers implementing
an accelerated move function.

This should keep the current behavior when removing the pre move wait.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It's pointless to only call the default implementation.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The function is a no-op with a NULL pointer.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

ttm_tt_destroy should be the only one unbinding the object.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It changes the way to skip newline character and also avoids
warning message from some compiler.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NEric Huang <JinHuiEric.Huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The soft pptable was used for re-uploading pptable as cache, but since
previous commits, the generic codes for uploading pptable are used and
backend is released during resetting powerplay. So it becomes redundance.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NEric Huang <JinHuiEric.Huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Due to uploading pptable implementation changed, the generic codes in
previous commit have been used intead of the Asic specific codes.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NEric Huang <JinHuiEric.Huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Necessary for re-initializing dpm with new pptables at runtime.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NEric Huang <JinHuiEric.Huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>