Allows us to set priorities in the scheduler.
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>

We only need to wait for jobs to be scheduled when
the dependency is from the same scheduler.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Before this patch the scheduler fence was created when we push the job
into the queue, so we could only get the fence after pushing it.

The mutex now was necessary to prevent the thread pushing the jobs to
the hardware from running faster than the thread pushing the jobs into
the queue.

Otherwise the thread pushing jobs into the queue would have accessed
possible freed up memory when it tries to get a reference to the fence.

So what you get in the end is thread A:
mutex_lock(&job->lock);
...
Kick of thread B.
...
mutex_unlock(&job->lock);

And thread B:
mutex_lock(&job->lock);
....
mutex_unlock(&job->lock);
kfree(job);

I'm actually not sure if I'm still up to date on this, but this usage
pattern used to be not allowed with mutexes. See here as well
https://lwn.net/Articles/575460/.

v2: remove unrelated changes, fix missing owner
v3: rebased, add more commit message
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Change-Id: I45bb8ff10ef05dc3b15e31a77fbcf31117705f11
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Change-Id: I67e987db0efdca28faa80b332b75571192130d33
Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NDavid Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Embed the scheduler into the ring structure instead of allocating it.
Use the ring name directly instead of the id.

v2: rebased, whitespace cleanup
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Chunming Zhou<david1.zhou@amd.com>

Just to be consistent with the other members.

v2: rename the ring member as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com> (v1)
Reviewed-by: Chunming Zhou<david1.zhou@amd.com>

Reorder the fields and properly return the kfifo_alloc error code.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Chunming Zhou<david1.zhou@amd.com>

Use consistent naming across functions.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NDavid Zhou <david1.zhou@amd.com>
Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>

Just free the resources immediately after submitting the job.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

And call the processed callback directly after submitting the job.

v2: split adding error handling into separate patch.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

This way the scheduler doesn't wait in it's work thread any more.

v2: fix race conditions
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

Freeing up a queue after signalling it isn't race free.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Removing the entity from scheduling can deadlock the whole system.
Wait forever till the remaining IBs are scheduled.

v2: fix comment as well
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Entity don't live as long as scheduler fences.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Calling schedule() is probably the worse things we can do.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

It isn't protecting anything.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Not used any more.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

That actually didn't worked at all.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

None of them are used any more.

v2: fix type in error message
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Remove active_hw_rq and it's protecting queue_lock, they are unused.

User 32bit atomic for hw_rq_count, 64bits for counting to three is a bit
overkill.

Cleanup the function name and remove incorrect comments.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>

Simply not used any more. Only keep 32bit atomic for fence sequence numbering.

v2: trivial rebase
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com> (v1)
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)

More appropriate and fixes some nasty lockdep warnings.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Rework run queue implementation, especially remove the odd list handling.

v2: cleanup the code only, no algorithem change.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

decrease CPU extra overhead.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Unused and missing proper locking.

v2: add locking comment to commit message.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)

Not used any more.

v2: remove amd_sched_emit as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

scheduler fence is based on kernel fence framework.

v2: squash in Christian's build fix
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

v2: rebased
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Avoiding a couple of casts.

v2: rename c_entity to entity as well
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

This reverts commit d1d33da8eb86b8ca41dd9ed95738030df5267b95.
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Conflicts:
	drivers/gpu/drm/amd/amdgpu/amdgpu_sched.c
	drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c

Id's are for the IOCTL ABI only.

v2: remove tgid as well
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>