Return proper pgprot for ARM64. This is required for objects like
Nouveau fences to be mapped with expected coherency.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Today, most callers of ttm_io_prot() check TTM_PL_FLAG_CACHED before
calling it since on some archs it will unconditionally create non-cached
mappings.

But not all callers do which is incorrect as far as I can tell.

Instead, move that check inside ttm_io_port() itself for all archs
and make powerpc use the same implementation as ia64 and arm
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>

No users are left, kill it off! :D
Conversion to the reservation api is next on the list, after
that the functionality can be restored with rcu.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>

Nouveau can now be used on ARM, so add an ioprot handler for this
architecture.
Signed-off-by: NLucas Stach <dev@lynxeye.de>
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NDavid Herrmann <dh.herrmann@gmail.com>

Mark functions as static because they are not used outside the file
drm/ttm/ttm_bo_util.c.

This eliminates the following warnings in drm/ttm/ttm_bo_util.c:
drivers/gpu/drm/ttm/ttm_bo_util.c:190:5: warning: no previous prototype for ‘ttm_mem_reg_ioremap’ [-Wmissing-prototypes]
drivers/gpu/drm/ttm/ttm_bo_util.c:222:6: warning: no previous prototype for ‘ttm_mem_reg_iounmap’ [-Wmissing-prototypes]
Signed-off-by: NRashika Kheria <rashika.kheria@gmail.com>
Reviewed-by: NJosh Triplett <josh@joshtriplett.org>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>

Commit "drm/ttm: Don't move non-existing data" didn't take the
swapped-out corner case into account. This patch corrects that.
Fixes blank screen after attempted suspend / hibernate on vmwgfx.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Checking directly for the right capability is simpler. Also this rids
us of a few places that use DRM_CURRENTPID.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

If ttm_bo_move_memcpy was instructed to move a non-populated ttm to
io memory, it would first populate the ttm, then move the data and then
destroy the ttm. That's stupid. However, some drivers might have relied on
this to clear io memory from old stuff. So instead of a NOP, which would
be the most efficient, just clear the destination.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: NJakob Bornecrantz <jakob@vmware.com>

All error paths will want to keep the mm node, so handle this at the
function exit. This fixes an ioremap failure error path.
Also add some comments to make the function a bit easier to understand.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: NJakob Bornecrantz <jakob@vmware.com>
Cc: stable@vger.kernel.org

Fix the case where the ttm pointer may be NULL causing
a NULL pointer dereference.
Signed-off-by: NJakob Bornecrantz <jakob@vmware.com>
Signed-off-by: NThomas Hellström <thellstrom@vmware.com>
Cc: stable@vger.kernel.org

Use the new vma-manager infrastructure. This doesn't change any
implementation details as the vma-offset-manager is nearly copied 1-to-1
from TTM.

The vm_lock is moved into the offset manager so we can drop it from TTM.
During lookup, we use the vma locking helpers to take a reference to the
found object.
In all other scenarios, locking stays the same as before. We always
guarantee that drm_vma_offset_remove() is called only during destruction.
Hence, helpers like drm_vma_node_offset_addr() are always safe as long as
the node has a valid offset.

This also drops the addr_space_offset member as it is a copy of vm_start
in vma_node objects. Use the accessor functions instead.

v4:
 - remove vm_lock
 - use drm_vma_offset_lock_lookup() to protect lookup (instead of vm_lock)

Cc: Dave Airlie <airlied@redhat.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Cc: Martin Peres <martin.peres@labri.fr>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDavid Herrmann <dh.herrmann@gmail.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@gmail.com>

Now that the code is compatible in semantics, flip the switch.
Use ww_mutex instead of the homegrown implementation.

ww_mutex uses -EDEADLK to signal that the caller has to back off,
and -EALREADY to indicate this buffer is already held by the caller.

ttm used -EAGAIN and -EDEADLK for those, respectively. So some changes
were needed to handle this correctly.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

qxl wants to use io mapping like i915 gem does, for now
just export the symbols so the driver can implement atomic
page maps using io mapping.
Signed-off-by: NDave Airlie <airlied@redhat.com>

This fixes up

commit e8e89622
Author: Daniel Vetter <daniel.vetter@ffwll.ch>
Date:   Tue Dec 18 22:25:11 2012 +0100

    drm/ttm: fix fence locking in ttm_buffer_object_transfer

which leaves behind a might_sleep in atomic context, since the
fence_lock spinlock is held over a kmalloc(GFP_KERNEL) call. The fix
is to revert the above commit and only take the lock where we need it,
around the call to ->sync_obj_ref.

v2: Fixup things noticed by Maarten Lankhorst:
- Brown paper bag locking bug.
- No need for kzalloc if we clear the entire thing on the next line.
- check for bo->sync_obj (totally unlikely race, but still someone
  else could have snuck in) and clear fbo->sync_obj if it's cleared
  already.
Reported-by: NDave Airlie <airlied@gmail.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

When we are using memcpy to move objects around, and we fail to memcpy
due to lack of memory to populate or failure to finish the copy, we don't
want to destroy the mm_node that has been copied into old_copy.

While working on a new kms driver that uses memcpy, if I overallocated bo's
up to the memory limits, and eviction failed, then machine would oops soon
after due to having an active bo with an already freed drm_mm embedded in it,
freeing it a second time didn't end well.
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Noticed while reviewing the fence locking in the radeon pageflip
handler.

v2: Instead of grabbing the bdev->fence_lock in object_transfer just
move the single callsite of that function a few lines, so that it is
protected by the fence_lock. Suggested by Jerome Glisse.

v3: Fix typo in commit message.
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

All items on the lru list are always reservable, so this is a stupid
thing to keep. Not only that, it is used in a way which would
guarantee deadlocks if it were ever to be set to block on reserve.

This is a lot of churn, but mostly because of the removal of the
argument which can be nested arbitrarily deeply in many places.

No change of code in this patch except removal of the no_wait_reserve
argument, the previous patch removed the use of no_wait_reserve.

v2:
 - Warn if -EBUSY is returned on reservation, all objects on the list
   should be reservable. Adjusted patch slightly due to conflicts.
v3:
 - Focus on no_wait_reserve removal only.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-By: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

vmwgfx was its only user and always sets it to the same..
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-By: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Convert #include "..." to #include <path/...> in drivers/gpu/.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
Acked-by: NDave Airlie <airlied@redhat.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: NDave Jones <davej@redhat.com>

Signed-off-by: NHuacai Chen <chenhc@lemote.com>
Signed-off-by: NHongliang Tao <taohl@lemote.com>
Signed-off-by: NHua Yan <yanh@lemote.com>
Cc: dri-devel@lists.freedesktop.org
Signed-off-by: NDave Airlie <airlied@redhat.com>

Provide helper function to compute the kernel memory size needed
for each buffer object. Move all the accounting inside ttm, simplifying
driver and avoiding code duplication accross them.

v2 fix accounting of ghost object, one would have thought that i
   would have run into the issue since a longtime but it seems
   ghost object are rare when you have plenty of vram ;)
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>

Move the page allocation and freeing to driver callback and
provide ttm code helper function for those.

Most intrusive change, is the fact that we now only fully
populate an object this simplify some of code designed around
the page fault design.

V2 Rebase on top of memory accounting overhaul
V3 New rebase on top of more memory accouting changes
V4 Rebase on top of no memory account changes (where/when is my
   delorean when i need it ?)
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>

This reverts commit dfadbbdb.

Further upstream discussion between Marek and Thomas decided this wasn't
fully baked and needed further work, so revert it before it hits mainline.
Signed-off-by: NDave Airlie <airlied@redhat.com>

Pointed out by Michel Daenzer.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Sometimes we want to know whether a buffer is busy and wait for it (bo_wait).
However, sometimes it would be more useful to be able to query whether
a buffer is busy and being either read or written, and wait until it's stopped
being either read or written. The point of this is to be able to avoid
unnecessary waiting, e.g. if a GPU has written something to a buffer and is now
reading that buffer, and a CPU wants to map that buffer for read, it needs to
only wait for the last write. If there were no write, there wouldn't be any
waiting needed.

This, or course, requires user space drivers to send read/write flags
with each relocation (like we have read/write domains in radeon, so we can
actually use those for something useful now).

Now how this patch works:

The read/write flags should passed to ttm_validate_buffer. TTM maintains
separate sync objects of the last read and write for each buffer, in addition
to the sync object of the last use of a buffer. ttm_bo_wait then operates
with one the sync objects.
Signed-off-by: NMarek Olšák <maraeo@gmail.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Nouveau makes the assumption that if a TTM is bound there will be a mm_node
around for it and the backwards ordering here resulted in a use-after-free
on some eviction paths.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Drivers using their own implementation of io_mem_reserve/io_mem_free are
likely to store the tracking information for the map in mem.mm_node, so
it can't be freed while still mapped.

Signed-off-by: Ben Skeggs<bskeggs@redhat.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This patch attempts to fix up shortcomings with the current calling
sequences.

1) There's a fastpath where no locking occurs and only io_mem_reserved is
   called to obtain needed info for mapping. The fastpath is set per
   memory type manager.
2) If the fastpath is disabled, io_mem_reserve and io_mem_free will be exactly
   balanced and not called recursively for the same struct ttm_mem_reg.
3) Optionally the driver can choose to enable a per memory type manager LRU
   eviction mechanism that, when io_mem_reserve returns -EAGAIN will attempt
   to kill user-space mappings of memory in that manager to free up needed
   resources
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

The bo lock used only to protect the bo sync object members, and since it
is a per bo lock, fencing a buffer list will see a lot of locks and unlocks.
Replace it with a per-device lock that protects the sync object members on
*all* bos. Reading and setting these members will always be very quick, so
the risc of heavy lock contention is microscopic. Note that waiting for
sync objects will always take place outside of this lock.

The bo device fence lock will eventually be replaced with a seqlock /
rcu mechanism so we can determine that a bo is idle under a
rcu / read seqlock.

However this change will allow us to batch fencing and unreserving of
buffers with a minimal amount of locking.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: NJerome Glisse <j.glisse@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Keep the current interface but ignore the KM_type and use a stack based
approach.

The advantage is that we get rid of crappy code like:

	#define __KM_PTE			\
		(in_nmi() ? KM_NMI_PTE : 	\
		 in_irq() ? KM_IRQ_PTE :	\
		 KM_PTE0)

and in general can stop worrying about what context we're in and what kmap
slots might be appropriate for that.

The downside is that FRV kmap_atomic() gets more expensive.

For now we use a CPP trick suggested by Andrew:

  #define kmap_atomic(page, args...) __kmap_atomic(page)

to avoid having to touch all kmap_atomic() users in a single patch.

[ not compiled on:
  - mn10300: the arch doesn't actually build with highmem to begin with ]

[akpm@linux-foundation.org: coding-style fixes]
[akpm@linux-foundation.org: fix up drivers/gpu/drm/i915/intel_overlay.c]
Acked-by: NRik van Riel <riel@redhat.com>
Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
Acked-by: NChris Metcalf <cmetcalf@tilera.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: David Miller <davem@davemloft.net>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Dave Airlie <airlied@linux.ie>
Cc: Li Zefan <lizf@cn.fujitsu.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Nouveau will need this on GeForce 8 and up to account for the GPU
reordering physical VRAM for some memory types.
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Acked-by: NThomas Hellström <thellstrom@vmware.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Existing core code/drivers call drm_mm_put_block on ttm_mem_reg.mm_node
directly.  Future patches will modify TTM behaviour in such a way that
ttm_mem_reg.mm_node doesn't necessarily belong to drm_mm.
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Acked-by: NThomas Hellström <thellstrom@vmware.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

It makes sense for a BO to move after a process has requested
exclusive RW access on it (e.g. because the BO used to be located in
unmappable VRAM and we intercepted the CPU access from the fault
handler).

If we let the ghost object inherit cpu_writers from the original
object, ttm_bo_release_list() will raise a kernel BUG when the ghost
object is destroyed. This can be reproduced with the nouveau driver on
nv5x.
Reported-by: NMarcin Slusarz <marcin.slusarz@gmail.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Tested-by: NMarcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Only ever assigned, never used.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
[glisse: I will re-add if needed for range-restricted allocations]
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDave Airlie <airlied@redhat.com>

If the memory is not iomem we should not try to
ioremap it. Should fix :

https://bugs.freedesktop.org/show_bug.cgi?id=27822Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Tested-by: NRafał Miłecki <zajec5@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

All TTM driver have been converted to new io_mem_reserve/free
interface which allow driver to choose and return proper io
base, offset to core TTM for ioremapping if necessary. This
patch remove what is now deadcode.

V2 adapt to match with change in first patch of the patchset
V3 update after io_mem_reserve/io_mem_free callback balancing
V4 adjust to minor cleanup
V5 remove the needs ioremap flag
V6 keep the ioremapping facility in TTM

[airlied- squashed driver removals in here also]
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

On fault the driver is given the opportunity to perform any operation
it sees fit in order to place the buffer into a CPU visible area of
memory. This patch doesn't break TTM users, nouveau, vmwgfx and radeon
should keep working properly. Future patch will take advantage of this
infrastructure and remove the old path from TTM once driver are
converted.

V2 return VM_FAULT_NOPAGE if callback return -EBUSY or -ERESTARTSYS
V3 balance io_mem_reserve and io_mem_free call, fault_reserve_notify
   is responsible to perform any necessary task for mapping to succeed
V4 minor cleanup, atomic_t -> bool as member is protected by reserve
   mecanism from concurent access
V5 the callback is now responsible for iomapping the bo and providing
   a virtual address this simplify TTM and will allow to get rid of
   TTM_MEMTYPE_FLAG_NEEDS_IOREMAP
V6 use the bus addr data to decide to ioremap or this isn't needed
   but we don't necesarily need to ioremap in the callback but still
   allow driver to use static mapping
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

There is case where we want to be able to wait only for the
GPU while not waiting for other buffer to be unreserved. This
patch split the no_wait argument all the way down in the whole
ttm path so that upper level can decide on what to wait on or
not.

[airlied: squashed these 4 for bisectability reasons.]
drm/radeon/kms: update to TTM no_wait splitted argument
drm/nouveau: update to TTM no_wait splitted argument
drm/vmwgfx: update to TTM no_wait splitted argument
[vmwgfx patch: Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>]
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Acked-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>