The purpose of pinning is to prevent a buffer from moving.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Tested-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

That avoids lock inversion between the BO reservation lock
and the anon_vma lock.

v2:
* Changed amdgpu_bo_list_entry.user_pages to an array of pointers
* Lock mmap_sem only for get_user_pages
* Added invalidation of unbound userpointer BOs
* Fixed memory leak and page reference leak

v3 (chk):
* Revert locking mmap_sem only for_get user_pages
* Revert adding invalidation of unbound userpointer BOs
* Sanitize and fix error handling

v4 (chk):
* Init userpages pointer everywhere.
* Fix error handling when get_user_pages() fails.
* Add invalidation of unbound userpointer BOs again.

v5 (chk):
* Add maximum number of tries.

v6 (chk):
* Fix error handling when we run out of tries.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> (v4)
Acked-by: NAlex Deucher <alexander.deucher@amd.com>

Remember the tasks which are inside get_user_pages()
and ignore MMU callbacks from there.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We need to use post-decrement to get the pci_map_page undone also for
i==0, and to avoid some very unpleasant behaviour if pci_map_page
failed already at i==0.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NRasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

We will soon modify the vanilla get_user_pages() so it can no
longer be used on mm/tasks other than 'current/current->mm',
which is by far the most common way it is called.  For now,
we allow the old-style calls, but warn when they are used.
(implemented in previous patch)

This patch switches all callers of:

	get_user_pages()
	get_user_pages_unlocked()
	get_user_pages_locked()

to stop passing tsk/mm so they will no longer see the warnings.
Signed-off-by: NDave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave@sr71.net>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: jack@suse.cz
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20160212210156.113E9407@viggo.jf.intel.comSigned-off-by: NIngo Molnar <mingo@kernel.org>

This allows us to remove the global kernel context.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Updates from different VMs can be processed independently.

v2: agd: rebase on upstream
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

No need to keep that for every IB.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add a job_alloc_with_ib helper and proper job submission.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We can't submit to multiple rings at the same time anyway.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The padding depends on the firmware version and we need that for BO moves as
well, not only for VM updates.

v2: new approach of making pad_ib a ring function
v3: fix typo in macro name
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of when we try to bind it check the usermm when
we try to use it in the IOCTLs.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>

Non-scheduler code is longer supported.

v2: agd: rebased on upstream
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NKen Wang  <Qingqing.Wang@amd.com>
Reviewed-by: NMonk Liu <monk.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Otherwise we could try to evict overlapping userptr BOs in get_user_pages(),
leading to a possible circular locking dependency.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>

Otherwise we could try to evict overlapping userptr BOs in get_user_pages(),
leading to a possible circular locking dependency.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NKen Wang <Qingqing.Wang@amd.com>
Cc: stable@vger.kernel.org

Not necessary for VRAM.

v2: no need to check if ttm is NULL.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Missing error check if the operation failed.
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Missing error check if the operation failed.
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

The amdgpu driver has a debugfs interface that shows the amount of
VRAM in use, but the newly added code causes a build error on
all 32-bit architectures:

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c:1076:17: warning: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'long long int' [-Wformat=]

This fixes the format string to use "%llu" for printing 64-bit
numbers, which works everywhere, as long as we also cast to 'u64'.
Unlike atomic64_t, u64 is defined as 'unsigned long long' on
all architectures.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Fixes: a2ef8a97 ("drm/amdgpu: add vram usage into debugfs")
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Moves are exclusive operations anyway, just use the undefined owner for those.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Adds an extra argument to amdgpu_bo_create, which is only used in amdgpu_prime.c.

Port of radeon commit 831b6966.

v2: fix up kfd.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

For kernel driver BOs, be explicit about whether we need
vram access up front.  This avoids unecessary migrations and
avoids using visible vram for buffers were it's not needed.

v2: line wrap fixes
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This aids handling buffers moves with the scheduler.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

when eviction is happening, if don't handle
dependency, then the fence could be dead off.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian K?nig <christian.koenig@amd.com>

Use kzalloc rather than kcalloc(1.. for allocating
one thing.
Signed-off-by: NManinder Singh <maninder1.s@samsung.com>
Reviewed-by: NVaneet Narang <v.narang@samsung.com>
Reviewed-by: NChristian Konig <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Not needed any more.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

sg could point to array of contigiouse page*, only free page could lead
to memory leak.

v2: use iterator
Signed-off-by: Nmonk.liu <monk.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This adds the non-asic specific core driver code.

v2: remove extra kconfig option
v3: implement minor fixes from Fengguang Wu
v4: fix cast in amdgpu_ucode.c
Acked-by: NChristian König <christian.koenig@amd.com>
Acked-by: NJammy Zhou <Jammy.Zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Fixing a memory leak with userptrs.

v2: clean up the loop, use an iterator instead
v3: remove unused variable
Signed-off-by: Nmonk.liu <monk.liu@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
CC: stable@vger.kernel.org
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We somehow try to free the SG table twice.

Bugs: https://bugs.freedesktop.org/show_bug.cgi?id=89734Signed-off-by: NChristian König <christian.koenig@amd.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Check the that ring we are using for copies is functional
rather than the GFX ring.  On newer asics we use the DMA
ring for bo moves.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Adds an extra argument to radeon_bo_create, which is only used in radeon_prime.c.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Really, the legacy buffer api should be dead, especially for all these
newfangled drivers. I suspect this is copypasta from the transitioning
days, which probably originated in radeon.

Cc: "Christian König" <christian.koenig@amd.com>
Cc: David Herrmann <dh.herrmann@gmail.com>
Cc: Rashika <rashika.kheria@gmail.com>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Fabian Frederick <fabf@skynet.be>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Alexandre Courbot <acourbot@nvidia.com>
Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Cc: Christian Engelmayer <cengelma@gmx.at>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Reviewed-by: NDavid Herrmann <dh.herrmann@gmail.com>
Acked-by: NBen Skeggs <bskeggs@redhat.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

AGP mappings are not cache coherent, so userptr support
won't work. Additional to that the AGP implementation uses
a different ttm_tt container structure so we run into
problems if we cast the pointer without checking if it's
the right type.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This allows us to specify if we want to sync to
the shared fences of a reservation object or not.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>