Add an aid for the driver to detect deadlocks on multi-bo reservations
Update documentation.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: NJerome Glisse <j.glisse@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Avoid the ttm_bo_unreserve() spinlocks by calling
ttm_eu_backoff_reservation_locked under the lru spinlock.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: NJerome Glisse <j.glisse@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Makes it possible to reserve a list of buffer objects with a single
spin lock / unlock if there is no contention.
Should improve cpu usage on SMP kernels.

v2: Initialize private list members on reserve and don't call
ttm_bo_list_ref_sub() with zero put_count.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

A process suspended waiting for a higher sequence or no sequence to unreserve,
a bo may be beaten to the reservation by a process with a lower sequence.
In that case the first process should give up trying to reserve and
return -EAGAIN. In order for that to happen, we must wake waiting processes
when we change sequence, so that they have a chance to detect the new
sequence.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Call destroy() on _all_ ttm_bo_init() failures, and make sure that
behavior is documented in the function description.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This breaks vmwgfx non-root EGL clients and is a remnant from the
TTM user-space interface. This test should be done in the driver.
Replace the remaining placement test with a BUG_ON, since triggering
it is a driver bug.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

The sync object may disappear as soon as we release the bo::lock, so
take a reference on it while we use it.
One option would be to call sync_object_flush() before releasing the bo::lock,
but that would put an atomic requirement on that function.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

The driver (for example vmwgfx) may want to silently deal with the
error itself.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Since we're doing this outside of a spinlock to provide the necessary
barriers, add an explicit barrier.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Replace with BUG_ON(). These error messages remained from the time
when TTM was initialized from user-space. Nowadays hitting one of those
is really a kernel bug.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Searching for a free block in the range manager may in some situations be a
lenghty operation, and we want to avoid holding the global lru lock
during that time. Instead use a per-manager spinlock.

This leaves the global lru lock for quick lru list and swap list manipulation
only, including list manipulation associated with reserving buffer objects.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Remove an obsolete comment about mm nodes.
Document the new bo range manager interface.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Keep the current interface but ignore the KM_type and use a stack based
approach.

The advantage is that we get rid of crappy code like:

	#define __KM_PTE			\
		(in_nmi() ? KM_NMI_PTE : 	\
		 in_irq() ? KM_IRQ_PTE :	\
		 KM_PTE0)

and in general can stop worrying about what context we're in and what kmap
slots might be appropriate for that.

The downside is that FRV kmap_atomic() gets more expensive.

For now we use a CPP trick suggested by Andrew:

  #define kmap_atomic(page, args...) __kmap_atomic(page)

to avoid having to touch all kmap_atomic() users in a single patch.

[ not compiled on:
  - mn10300: the arch doesn't actually build with highmem to begin with ]

[akpm@linux-foundation.org: coding-style fixes]
[akpm@linux-foundation.org: fix up drivers/gpu/drm/i915/intel_overlay.c]
Acked-by: NRik van Riel <riel@redhat.com>
Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
Acked-by: NChris Metcalf <cmetcalf@tilera.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: David Miller <davem@davemloft.net>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Dave Airlie <airlied@linux.ie>
Cc: Li Zefan <lizf@cn.fujitsu.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This commit replaces the ttm_bo_cleanup_ref function with two new functions.
One for the case where the bo is not yet on the delayed destroy list, and
one for the case where the bo was on the delayed destroy list, at least at
the time of call. This makes it possible to optimize the two cases somewhat.

It also enables the possibility to directly destroy buffers on the
delayed delete list when they are about to be evicted or swapped out.
Currently they were only evicted / swapped and destruction was left for the
delayed buffer destruction thread.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Release the lru spinlock early.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Function ttm_bo_wait_unreserved can be slightly simplified.
Signed-off-by: NJean Delvare <khali@linux-fr.org>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

We need the unlocked variant for the new codepath introduced to fix the
race condition in master recently.
Signed-off-by: NDave Airlie <airlied@redhat.com>

This fixes a race pointed out by Dave Airlie where we don't take a buffer
object about to be destroyed off the LRU lists properly. It also fixes a rare
case where a buffer object could be destroyed in the middle of an
accelerated eviction.

The patch also adds a utility function that can be used to prematurely
release GPU memory space usage of an object waiting to be destroyed.
For example during eviction or swapout.

The above mentioned commit didn't queue the buffer on the delayed destroy
list under some rare circumstances. It also didn't completely honor the
remove_all parameter.

Fixes:
https://bugzilla.redhat.com/show_bug.cgi?id=615505
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=591061Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Nouveau will need this on GeForce 8 and up to account for the GPU
reordering physical VRAM for some memory types.
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Acked-by: NThomas Hellström <thellstrom@vmware.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Existing core code/drivers call drm_mm_put_block on ttm_mem_reg.mm_node
directly.  Future patches will modify TTM behaviour in such a way that
ttm_mem_reg.mm_node doesn't necessarily belong to drm_mm.
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Acked-by: NThomas Hellström <thellstrom@vmware.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Correct allocation flags type and function prototype for ANSI C compliance.

[airlied: whitespace fixed]
Signed-off-by: NDaniel J Blueman <daniel.blueman@gmail.com>
Reviewed-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

It makes sense for a BO to move after a process has requested
exclusive RW access on it (e.g. because the BO used to be located in
unmappable VRAM and we intercepted the CPU access from the fault
handler).

If we let the ghost object inherit cpu_writers from the original
object, ttm_bo_release_list() will raise a kernel BUG when the ghost
object is destroyed. This can be reproduced with the nouveau driver on
nv5x.
Reported-by: NMarcin Slusarz <marcin.slusarz@gmail.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Tested-by: NMarcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Signed-off-by: NDave Airlie <airlied@redhat.com>

I wrote this for the prime sharing work, but I also noticed other external
non-upstream drivers from a large company carrying a similiar patch, so I
may as well ship it in master.
Signed-off-by: NDave Airlie <airlied@redhat.com>

The commit 1e8655f8
    drm/ttm: Fix build on architectures without AGP

looks at TTM_HAS_AGP before it has been set in ttm_bo_driver.h

Move the conditional inclusion of <asm/agp.h> *after* we have included
ttm_bo_driver.h
Signed-of-by: NTony Luck <tony.luck@intel.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Add the shrinkers missed in the first conversion of the API in
commit 7f8275d0 ("mm: add context argument to
shrinker callback").
Signed-off-by: NDave Chinner <dchinner@redhat.com>

Make inclusion of <asm/agp.h> conditional on TTM_HAS_AGP.  The use
of the functions declared in it is already conditional.
Reported-by: NGeert Stappers <stappers@stappers.nl>
Signed-off-by: NBen Hutchings <ben@decadent.org.uk>
Tested-by: NGeert Stappers <stappers@stappers.nl>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Only ever assigned, never used.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
[glisse: I will re-add if needed for range-restricted allocations]
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Repeated ttm_page_alloc_init/fini fails noisily because the pool
manager kobj isn't zeroed out between uses (we could do just that but
statically allocated kobjects are generally considered a bad thing).
Move it to kzalloc'ed memory.

Note that this patch drops the refcounting behavior of the pool
allocator init/fini functions: it would have led to a race condition
in its current form, and anyway it was never exploited.

This fixes a regression with reloading kms modules at runtime, since
page allocator was introduced.
Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Non pooled page allocation should have GFP_USER set so allocation
can wait and reclaim page from other process (ie non atomic).
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Fix a number of typos misspellings and checkpatch.pl warnings.
Replace "[ttm] " with TTM_PFX
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This patch fixes a regression introduced with the pool page allocator
in the event that there are no highmem pages (for example x86_64),
in which case cached page allocation would fail.

Tested with the vmwgfx driver on a 64-bit vm.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

We want to be able to prevent the delayed workqueue from changing state
while we're reclocking, so add an API to block and unblock it.
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

It's unused and buggy in its current form, since it can place a bo
in the reserved state without removing it from lru lists.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

If the memory is not iomem we should not try to
ioremap it. Should fix :

https://bugs.freedesktop.org/show_bug.cgi?id=27822Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Tested-by: NRafał Miłecki <zajec5@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

All TTM driver have been converted to new io_mem_reserve/free
interface which allow driver to choose and return proper io
base, offset to core TTM for ioremapping if necessary. This
patch remove what is now deadcode.

V2 adapt to match with change in first patch of the patchset
V3 update after io_mem_reserve/io_mem_free callback balancing
V4 adjust to minor cleanup
V5 remove the needs ioremap flag
V6 keep the ioremapping facility in TTM

[airlied- squashed driver removals in here also]
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

On fault the driver is given the opportunity to perform any operation
it sees fit in order to place the buffer into a CPU visible area of
memory. This patch doesn't break TTM users, nouveau, vmwgfx and radeon
should keep working properly. Future patch will take advantage of this
infrastructure and remove the old path from TTM once driver are
converted.

V2 return VM_FAULT_NOPAGE if callback return -EBUSY or -ERESTARTSYS
V3 balance io_mem_reserve and io_mem_free call, fault_reserve_notify
   is responsible to perform any necessary task for mapping to succeed
V4 minor cleanup, atomic_t -> bool as member is protected by reserve
   mecanism from concurent access
V5 the callback is now responsible for iomapping the bo and providing
   a virtual address this simplify TTM and will allow to get rid of
   TTM_MEMTYPE_FLAG_NEEDS_IOREMAP
V6 use the bus addr data to decide to ioremap or this isn't needed
   but we don't necesarily need to ioremap in the callback but still
   allow driver to use static mapping
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NDave Airlie <airlied@redhat.com>