scheduler fence is based on kernel fence framework.

v2: squash in Christian's build fix
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Avoiding a couple of casts.

v2: rename c_entity to entity as well
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Cleanup the kernel context handling.

v2: rebased
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)

Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

We didn't initialized the mutex in the cloned bo list resulting in nice
warnings from lockdep. Also fixes error handling in this function.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

This way can avoid interrupt lost, and can process sched job exactly.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

every sbumission should be able to get a fence.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <jammy.zhou@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <jammy.zhou@amd.com>

It is theoretically possible that a swapped out BO gets the
same GTT address, but different backing pages while being swapped in.

Instead just use another VA state to note updated areas.
Ported from not upstream yet radeon commit with the same name.

v2: fix some bugs in the original implementation found in the radeon code.
v3: squash in VCE/UVD fix
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

thus unnecessary wake up could be avoid between rings
v2:
move wait_queue_head to fence_drv from ring
Signed-off-by: Nmonk.liu <monk.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

origninal method will sleep/schedule at the granurarity of HZ/2 and
based on seq signal method, the new implement is based on kernel fance
interface, no unnecessary schedule at all

v2: replace logic of original amdgpu_fence_wait_any
Signed-off-by: Nmonk.liu <monk.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

if enabling scheduler, then the queued seq is assigned
when pushing job before emitting job.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

This option can be used to specify the max number of submissions in the
active HW queue. The default value is 2 now.
Signed-off-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

This option can be used to specify the max job number in the job queue,
and it is 16 by default.
Signed-off-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

fence_process may be called from kthread, user thread and interrupt context.
it is possible to called concurrently, then will wake up fence queue multiple times.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

use kernel context to submit command for vm
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

v2: rebase against kfd changes
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

user mode will still use pte ring as a normal ring.
if the prepare job generates another command(update pte) on its ring in scheduler,
then will kill scheduler which is going to waiting later job but pending running job.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

BO validation is moved to scheduler except usrptr which must be validated
in user process
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

v2: fix rebase breakage
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

1. Add kernel parameter option, default 0
2. Add scheduler initialization for amdgpu
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Signed-off-by: NJammy Zhou <Jammy.Zhou@amd.com>
Acked-by: NChristian K?nig <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

Make the definitions common for all driver components

v2: fix kfd
Signed-off-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

We need to return the sequence number to userspace
even when we don't use user fences.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Added to:
- handle draining the ring on suspend
- properly enable/disable interrupts on suspend and resume

Fix breakages from:
commit 467ee3be53d240d08beed2e82a941e820c1ac323
Author: Chunming Zhou <david1.zhou@amd.com>
Date:   Mon Jun 1 14:14:32 2015 +0800

    drm/amdgpu: always enable EOP interrupt v2
Tested-by: NAudrey Grodzovsky <audrey.grodzovsky@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2 (chk): always enable EOP interrupt, independent of scheduler,
	  remove now unused delayed_irq handling.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

This makes assigning VM IDs independent from the use of VM IDs.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

It's not used any more.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

This is a prerequisite for the GPU scheduler to make the order
of submission independent from the order of execution.

v2: properly implement the locking
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

This also requires some error handling from the callers of that function.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

The comment is misleading and incorrect, remove it.

Printing the id is completely meaningless and this practice
can cause a race conditions on command submission.

The flags and hangs fields are completely unused.

Give all fields a common indentation.

v2: remove fpriv reference and unused flags as well, fix debug message.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

CGS (Common Graphics Services) is an AMD cross component
abstraction layer to designed to better encapsulate
specific IP block drivers so different teams can effectively
work on differnet IP block drivers independently. It provides
a common interface for things like accessing registers,
allocating GPU memory, and registering interrupt sources.
The plan is to eventually move more and more IP drivers to
this interface.  The first user is the ACP IP driver.
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Expose feature version to user space for RLC/MEC/MEC2 ucode as well

v2: fix coding style
Signed-off-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

If we fail during device init, record what state each
block is in so that we can tear down clearly.

Fixes various problems on device init failure.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>