Extern is the default for function declerations anyway and this
solves a bunch of 80char per line issues.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This allows the driver to handle io_mem mappings on their own.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This variant of ttm_bo_init returns the validated buffer object with
the reservation lock held when resv == NULL. This is convenient for
callers that want to use the BO immediately, e.g. for initializing its
contents.
Signed-off-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

As the comment says: callers of ttm_bo_init cannot rely on having the
only reference to the BO when the function returns successfully.
Signed-off-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This way the driver can specify a priority for a BO which has the effect that
a BO is only evicted when all other BOs with a lower priority are evicted
first.
Reviewed-by: NSinclair Yeh <syeh@vmware.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NRoger.He <Hongbo.He@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

By general sentiment kref_sub() is a bad interface, make it go away.
Signed-off-by: NPeter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: NIngo Molnar <mingo@kernel.org>

This way the driver can decide if it is valuable to evict a BO or not.

The current implementation is added as default to all existing drivers.

v2: fix some typos found during internal testing
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

I plan to usurp the short name of struct fence for a core kernel struct,
and so I need to rename the specialised fence/timeline for DMA
operations to make room.

A consensus was reached in
https://lists.freedesktop.org/archives/dri-devel/2016-July/113083.html
that making clear this fence applies to DMA operations was a good thing.
Since then the patch has grown a bit as usage increases, so hopefully it
remains a good thing!

(v2...: rebase, rerun spatch)
v3: Compile on msm, spotted a manual fixup that I broke.
v4: Try again for msm, sorry Daniel

coccinelle script:
@@

@@
- struct fence
+ struct dma_fence
@@

@@
- struct fence_ops
+ struct dma_fence_ops
@@

@@
- struct fence_cb
+ struct dma_fence_cb
@@

@@
- struct fence_array
+ struct dma_fence_array
@@

@@
- enum fence_flag_bits
+ enum dma_fence_flag_bits
@@

@@
(
- fence_init
+ dma_fence_init
|
- fence_release
+ dma_fence_release
|
- fence_free
+ dma_fence_free
|
- fence_get
+ dma_fence_get
|
- fence_get_rcu
+ dma_fence_get_rcu
|
- fence_put
+ dma_fence_put
|
- fence_signal
+ dma_fence_signal
|
- fence_signal_locked
+ dma_fence_signal_locked
|
- fence_default_wait
+ dma_fence_default_wait
|
- fence_add_callback
+ dma_fence_add_callback
|
- fence_remove_callback
+ dma_fence_remove_callback
|
- fence_enable_sw_signaling
+ dma_fence_enable_sw_signaling
|
- fence_is_signaled_locked
+ dma_fence_is_signaled_locked
|
- fence_is_signaled
+ dma_fence_is_signaled
|
- fence_is_later
+ dma_fence_is_later
|
- fence_later
+ dma_fence_later
|
- fence_wait_timeout
+ dma_fence_wait_timeout
|
- fence_wait_any_timeout
+ dma_fence_wait_any_timeout
|
- fence_wait
+ dma_fence_wait
|
- fence_context_alloc
+ dma_fence_context_alloc
|
- fence_array_create
+ dma_fence_array_create
|
- to_fence_array
+ to_dma_fence_array
|
- fence_is_array
+ dma_fence_is_array
|
- trace_fence_emit
+ trace_dma_fence_emit
|
- FENCE_TRACE
+ DMA_FENCE_TRACE
|
- FENCE_WARN
+ DMA_FENCE_WARN
|
- FENCE_ERR
+ DMA_FENCE_ERR
)
 (
 ...
 )
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NGustavo Padovan <gustavo.padovan@collabora.co.uk>
Acked-by: NSumit Semwal <sumit.semwal@linaro.org>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20161025120045.28839-1-chris@chris-wilson.co.uk

Makes more sense to keep that together.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of using the flag just remember the fence of the last move operation.

This avoids waiting for command submissions pipelined after the move, but
before accessing the BO with the CPU again.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

There are cases where it is desired to see if a proposed placement
is compatible with a buffer object before calling ttm_bo_validate().
Signed-off-by: NSinclair Yeh <syeh@vmware.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Cc: <stable@vger.kernel.org>
---
This is the first of a 3-patch series to fix a black screen
issue observed on Ubuntu 16.04 server.

Not used any more.
Reviewed-by: NSinclair Yeh <syeh@vmware.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Fixes ttm on platforms like PPC460 where the CPU
is in 32-bit mode, but the physical addresses are
>32 bits.

Extracted from a patch by Hans Verkuil.
Tested-by: NJulian Margetson <runaway@candw.ms>
Acked-by: NThomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Cc: Julian Margetson <runaway@candw.ms>
Cc: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This allows the drivers to move a BO to the end of the LRU
without removing and adding it again.

v2: Make it more robust, handle pinned and swapable BOs as well.
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We need to store device offsets in 64 bit as the device
address space may be larger than the CPU's.

Fixes GPU init failures on radeons with 4GB or more of
vram on 32 bit kernels.  We put vram at the start of the
GPU's address space so the gart aperture starts at 4 GB
causing all GPU addresses in the gart aperture to get
truncated.

bug:
https://bugs.freedesktop.org/show_bug.cgi?id=89072

[airlied: fix warning on nouveau build]
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: thellstrom@vmware.com
Acked-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This allows importing reservation objects from dma-bufs.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>

Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>

No users are left, kill it off! :D
Conversion to the reservation api is next on the list, after
that the functionality can be restored with rcu.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>

This allows us to more fine grained specify where to place the buffer object.

v2: rebased on drm-next, add bochs changes as well
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

The kerneldoc header of ttm_bo_create() was referring to another
(nonexisting) function and had a few obsolete or incorrect arguments.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Reviewed-by: NDavid Herrmann <dh.herrmann@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Addresses
"[BUG] completely bonkers use of set_need_resched + VM_FAULT_NOPAGE".

In the first occurence it was used to try to be nice while releasing the
mmap_sem and retrying the fault to work around a locking inversion.
The second occurence was never used.

There has been some discussion whether we should change the locking order to
mmap_sem -> bo_reserve. This patch doesn't address that issue, and leaves
that locking order undefined. The solution that we release the mmap_sem if
tryreserve fails and wait for the buffer to become unreserved is something
we want in any case, and follows how the core vm system waits for pages
to be come unlocked while releasing the mmap_sem.

The code also outlines what needs to be changed if we want to establish the
locking order as mmap_sem -> bo::reserve.

One slight issue that remains with this code is that the fault handler might
be prone to starvation if another thread countinously reserves the buffer.
IMO that usage pattern is highly unlikely.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>

Use the new vma-manager infrastructure. This doesn't change any
implementation details as the vma-offset-manager is nearly copied 1-to-1
from TTM.

The vm_lock is moved into the offset manager so we can drop it from TTM.
During lookup, we use the vma locking helpers to take a reference to the
found object.
In all other scenarios, locking stays the same as before. We always
guarantee that drm_vma_offset_remove() is called only during destruction.
Hence, helpers like drm_vma_node_offset_addr() are always safe as long as
the node has a valid offset.

This also drops the addr_space_offset member as it is a copy of vm_start
in vma_node objects. Use the accessor functions instead.

v4:
 - remove vm_lock
 - use drm_vma_offset_lock_lookup() to protect lookup (instead of vm_lock)

Cc: Dave Airlie <airlied@redhat.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Cc: Martin Peres <martin.peres@labri.fr>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDavid Herrmann <dh.herrmann@gmail.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@gmail.com>

Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Now that the code is compatible in semantics, flip the switch.
Use ww_mutex instead of the homegrown implementation.

ww_mutex uses -EDEADLK to signal that the caller has to back off,
and -EALREADY to indicate this buffer is already held by the caller.

ttm used -EAGAIN and -EDEADLK for those, respectively. So some changes
were needed to handle this correctly.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This commit converts the source of the val_seq counter to
the ww_mutex api. The reservation objects are converted later,
because there is still a lockdep splat in nouveau that has to
resolved first.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

All items on the lru list are always reservable, so this is a stupid
thing to keep. Not only that, it is used in a way which would
guarantee deadlocks if it were ever to be set to block on reserve.

This is a lot of churn, but mostly because of the removal of the
argument which can be nested arbitrarily deeply in many places.

No change of code in this patch except removal of the no_wait_reserve
argument, the previous patch removed the use of no_wait_reserve.

v2:
 - Warn if -EBUSY is returned on reservation, all objects on the list
   should be reservable. Adjusted patch slightly due to conflicts.
v3:
 - Focus on no_wait_reserve removal only.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This is similar to other platforms that don't allow command submission
to buffers locked on the cpu.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

vmwgfx was its only user and always sets it to the same..
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-By: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

All drivers set it to 0 and nothing uses it.
Signed-off-by: NMarcin Slusarz <marcin.slusarz@gmail.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Convert #include "..." to #include <path/...> in kernel system headers.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: NDave Jones <davej@redhat.com>

This adds the ability for ttm common code to take an SG table
and use it as the backing for a slave TTM object.

The drivers can then populate their GTT tables using the SG object.

v2: make sure to setup VM for sg bos as well.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Provide helper function to compute the kernel memory size needed
for each buffer object. Move all the accounting inside ttm, simplifying
driver and avoiding code duplication accross them.

v2 fix accounting of ghost object, one would have thought that i
   would have run into the issue since a longtime but it seems
   ghost object are rare when you have plenty of vram ;)
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>

This was never use in none of the driver, properly using userspace
page for bo would need more code (vma interaction mostly). Removing
this dead code in preparation of ttm_tt & backend merge.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>

This reverts commit dfadbbdb.

Further upstream discussion between Marek and Thomas decided this wasn't
fully baked and needed further work, so revert it before it hits mainline.
Signed-off-by: NDave Airlie <airlied@redhat.com>

Sometimes we want to know whether a buffer is busy and wait for it (bo_wait).
However, sometimes it would be more useful to be able to query whether
a buffer is busy and being either read or written, and wait until it's stopped
being either read or written. The point of this is to be able to avoid
unnecessary waiting, e.g. if a GPU has written something to a buffer and is now
reading that buffer, and a CPU wants to map that buffer for read, it needs to
only wait for the last write. If there were no write, there wouldn't be any
waiting needed.

This, or course, requires user space drivers to send read/write flags
with each relocation (like we have read/write domains in radeon, so we can
actually use those for something useful now).

Now how this patch works:

The read/write flags should passed to ttm_validate_buffer. TTM maintains
separate sync objects of the last read and write for each buffer, in addition
to the sync object of the last use of a buffer. ttm_bo_wait then operates
with one the sync objects.
Signed-off-by: NMarek Olšák <maraeo@gmail.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

. and some comments to make it easier to understand.

Ackedby: Randy Dunlap <randy.dunlap@oracle.com>
[v2: Added some more updates from Randy Dunlap]
Signed-off-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NJan Engelhardt <jengelh@medozas.de>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Fixes generated by 'codespell' and manually reviewed.
Signed-off-by: NLucas De Marchi <lucas.demarchi@profusion.mobi>

This patch attempts to fix up shortcomings with the current calling
sequences.

1) There's a fastpath where no locking occurs and only io_mem_reserved is
   called to obtain needed info for mapping. The fastpath is set per
   memory type manager.
2) If the fastpath is disabled, io_mem_reserve and io_mem_free will be exactly
   balanced and not called recursively for the same struct ttm_mem_reg.
3) Optionally the driver can choose to enable a per memory type manager LRU
   eviction mechanism that, when io_mem_reserve returns -EAGAIN will attempt
   to kill user-space mappings of memory in that manager to free up needed
   resources
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>