Instead of when we try to bind it check the usermm when
we try to use it in the IOCTLs.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>

Move that out of the main IOCTL function.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Non-scheduler code is longer supported.

v2: agd: rebased on upstream
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NKen Wang  <Qingqing.Wang@amd.com>
Reviewed-by: NMonk Liu <monk.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The other ones don't have any VAs assigned anyway or are uninteresting to us.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Make UVD/VCE VM emulation more efficient.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Stop copying that to the bo list entry, it doesn't change anyway.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Move that into the BO list. No functional change.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Rename it to amdgpu_cs_parser_bos and move validation and bo list init there.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Remove unused user_ptr field, group fields by usage.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

No need to actually check the current placement. Just use the allowed domains
when the threshold is reached.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The threshold should only be computed once.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This fixes a race condition in the error case: since the pt bos have not
necessarily been reserved in case of an error, we could move a pt bo that
is currently in the middle of being evicted/moved by another process,
which then resulted in a BUG_ON in ttm_bo_add_to_lru.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Most VM BOs end up in the duplicates list, validate it
first make -ENOMEM less likely.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <David1.Zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This makes it less likely to run into an ENOMEM because
VM page tables are evicted last.

v2: move the BOs in the LRU tail after validation
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This avoids allocating it on the fly.

v2: fix grammar in comment
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

This way we avoid the extra allocation for the page directory entry.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

They share the reservation object with the page directory anyway.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

This fixes a random corruption under memory pressure. We need to fence
the BO for the user fence as well, otherwise it might be swapped out
and the GPU could write the fence value to an undesired location.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

No need for a GEM reference here.
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Unify the two code path again, cause they do pretty much the same thing.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <davdi1.zhou@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>

We need to clear parser.ibs and num_ibs before amd_sched_fence_create,
otherwise the IB could be freed twice if fence creates fails.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>

Before this patch the scheduler fence was created when we push the job
into the queue, so we could only get the fence after pushing it.

The mutex now was necessary to prevent the thread pushing the jobs to
the hardware from running faster than the thread pushing the jobs into
the queue.

Otherwise the thread pushing jobs into the queue would have accessed
possible freed up memory when it tries to get a reference to the fence.

So what you get in the end is thread A:
mutex_lock(&job->lock);
...
Kick of thread B.
...
mutex_unlock(&job->lock);

And thread B:
mutex_lock(&job->lock);
....
mutex_unlock(&job->lock);
kfree(job);

I'm actually not sure if I'm still up to date on this, but this usage
pattern used to be not allowed with mutexes. See here as well
https://lwn.net/Articles/575460/.

v2: remove unrelated changes, fix missing owner
v3: rebased, add more commit message
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

OGL needs these tracepoints to investigate performance issue.

Change-Id: I5e58187d061253f7d665dfce8e4e163ba91d3e2b
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>

No need any more to allocate that structure dynamically, just put it on the
stack. This is a start to cleanup some of the scheduler fallouts.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Fixing a memory leak when the scheduler is enabled.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

We recently changed the locking in this function and now there is a
missing unlock on error.  Also there are some other resources that we
should probably release as well...

Fixes: f48b2659 ('drm/amdgpu: fix the broken vm->mutex V2')
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>

fix the vm->mutex and ww_mutex confilcts.
vm->mutex is always token first, then ww_mutex.

V2: remove unneccessary checking for pt bo.

Change-Id: Iea56e183752c02831126d06d2f5b7a474a6e4743
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Finally getting rid of it.
Signed-off-by: NChristian König <christian.koenig@amd.com>

This makes it much easier to find when userspace misses to send some buffers.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Rework the sdma structures in the driver to
consolidate all of the sdma info into a single
structure and allow for asics that may have
different numbers of sdma instances.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The new amdgpu driver passes a user space pointer in a 64-bit structure
member, which is the correct way to do it, but it attempts to
directly cast it to a __user pointer in the kernel, which causes
a warning in three places:

drm/amd/amdgpu/amdgpu_cs.c: In function 'amdgpu_cs_parser_init':
drm/amd/amdgpu/amdgpu_cs.c:180:21: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
  chunk_array_user = (uint64_t __user *)(cs->in.chunks);

This changes all three to add an intermediate cast to 'unsigned long'
as other drivers do. This avoids the warning and works correctly on
both 32-bit and 64-bit architectures.

Fixes: e60b344f ("drm/amdgpu: optimize amdgpu_parser_init")
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The "i" variable should be signed or it leads to a crash in the error
handling code.

Fixes: 1d263474 ('drm/amdgpu: unwind properly in amdgpu_cs_parser_init()')
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The amdgpu_cs_parser_init() function doesn't clean up after itself but
instead the caller uses a free everything function amdgpu_cs_parser_fini()
on failure.  This style of error handling is often buggy.  In this
example, we call "drm_free_large(parser->chunks[i].kdata);" when it is
an unintialized pointer or when "parser->chunks" is NULL.

I fixed this bug by adding unwind code so that it frees everything that
it allocates.

I also mode some other very minor changes:
1) Renamed "r" to "ret".
2) Moved the chunk_array allocation to the start of the function.
3) Removed some initializers which are no longer needed.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reported-by: NIlja Van Sprundel <ivansprundel@ioactive.com>
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Embed the scheduler into the ring structure instead of allocating it.
Use the ring name directly instead of the id.

v2: rebased, whitespace cleanup
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Chunming Zhou<david1.zhou@amd.com>

Just to be consistent with the other members.

v2: rename the ring member as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com> (v1)
Reviewed-by: Chunming Zhou<david1.zhou@amd.com>

Use container_of rather than casting.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NDavid Zhou <david1.zhou@amd.com>
Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>

Use consistent naming across functions.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NDavid Zhou <david1.zhou@amd.com>
Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>

This allows for multiple BOs to have the same reservation object.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Some buffers (UVD/VM page tables) must be placed in VRAM,
but the byte restriction for moving buffers didn't took this
into account.

Port of radeon commit 4b095566.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>