Some buffers (UVD/VM page tables) must be placed in VRAM,
but the byte restriction for moving buffers didn't took this
into account.

Port of radeon commit 4b095566.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Calling schedule() is probably the worse things we can do.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Simply not used any more. Only keep 32bit atomic for fence sequence numbering.

v2: trivial rebase
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com> (v1)
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)

Stop double freeing the the BO list by pulling the content
of amdgpu_cs_parser_prepare_job() into the IOCTL function again.

v2: better commit message
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

The problem now is that we don't necessarily call amdgpu_ib_get()
in some error paths and so work with uninitialized data.

Better require that the memory is already zeroed.

v2: better commit message
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Pagetables can be moved and therefore the page directory update can be necessary
for the current cs even if none of the the bo's are moved. In that scenario
there is no fence between the sdma0 and gfx ring, so we add one.

v2 (chk): rebased
Signed-off-by: NBas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

bo_list_clone() will take a lot of time when bo_list hold too much
elements, like above 7000
Signed-off-by: NMonk.Liu <monk.liu@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NJammy Zhou <jammy.zhou@amd.com>

fix fence is released when pass to **fence sometimes.
add reference for it.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

scheduler fence is based on kernel fence framework.

v2: squash in Christian's build fix
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Avoiding a couple of casts.

v2: rename c_entity to entity as well
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

This reverts commit d1d33da8eb86b8ca41dd9ed95738030df5267b95.
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Conflicts:
	drivers/gpu/drm/amd/amdgpu/amdgpu_sched.c
	drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c

We didn't initialized the mutex in the cloned bo list resulting in nice
warnings from lockdep. Also fixes error handling in this function.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

It is clean to update last_queued_v_seq in the scheduler module
Signed-off-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Fix the code alignment, etc.

v2: rebase the code
Signed-off-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <jammy.zhou@amd.com>

It is theoretically possible that a swapped out BO gets the
same GTT address, but different backing pages while being swapped in.

Instead just use another VA state to note updated areas.
Ported from not upstream yet radeon commit with the same name.

v2: fix some bugs in the original implementation found in the radeon code.
v3: squash in VCE/UVD fix
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

if enabling scheduler, then the queued seq is assigned
when pushing job before emitting job.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

it is possible that the callback isn't defined sometimes.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

user mode will still use pte ring as a normal ring.
if the prepare job generates another command(update pte) on its ring in scheduler,
then will kill scheduler which is going to waiting later job but pending running job.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

BO validation is moved to scheduler except usrptr which must be validated
in user process
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

use kmalloc_array instead of kcalloc where appropriate and other
cleanups.
Signed-off-by: Nmonk.liu <monk.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

We need to return the sequence number to userspace
even when we don't use user fences.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

This is a prerequisite for the GPU scheduler to make the order
of submission independent from the order of execution.

v2: properly implement the locking
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

This also requires some error handling from the callers of that function.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

Just to make sure userspace don't send nonsense to the kernel.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

v2: remove unrelated whitespace change, fix C comment
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

And use common fence infrastructure for the wait.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>