Like the GTT pwrite path fix, this uses an optimistic path and a
fallback to get_user_pages.  Note that this means we have to stop using
vfs_write and roll it ourselves.
Signed-off-by: NEric Anholt <eric@anholt.net>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>

We've wanted this for a few consumers that touch the pages directly (such as
the following commit), which have been doing the refcounting outside of
get/put pages.
Signed-off-by: NEric Anholt <eric@anholt.net>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Since the pagefault path determines that the lock order we use has to be
mmap_sem -> struct_mutex, we can't allow page faults to occur while the
struct_mutex is held.  To fix this in pwrite, we first try optimistically to
see if we can copy from user without faulting.  If it fails, fall back to
using get_user_pages to pin the user's memory, and map those pages
atomically when copying it to the GPU.
Signed-off-by: NEric Anholt <eric@anholt.net>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The last 8 fence registers sit at a different offset, so when we went to set
fence number 8 in the lower offset, we instead set PGETBL_CTL, and the GPU
got all sorts of angry at us.

fd.o bug #20567.  Easily reproducible by running glxgears and killing it about
6 times.
Signed-off-by: NEric Anholt <eric@anholt.net>

The i915 also uses the fence registers for GPU access to tiled buffers so
we cannot reallocate one whilst it is on the active list. By performing a
LRU scan of the fenced buffers we also avoid waiting the possibility of
waiting on a pinned, or otherwise unusable, buffer.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

We need to check and report if there are no available fences - or else we
spin endlessly waiting for a buffer to magically unpin itself.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

As we may steal the fence register of an unpinned buffer for another,
every time we repin the buffer we need to recheck whether it needs to be
allocated a fence.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

If we wait upon a request and successfully unbind a buffer occupying a
fence register, then that slot will be freed and cause a NULL derefrence
upon rescanning.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

Prevents formatting nasty as below:

[drm:i915_gem_object_pin] *ERROR* Failure to bind: -12<3>[drm:i915_gem_evict_something] *ERROR* inactive empty 1 request empty 1 flushing empty 1
Signed-off-by: NKyle McMartin <kyle@redhat.com>
Signed-off-by: NEric Anholt <eric@anholt.net>

If userspace passes an object list with the same object appearing more
than once, we end up hitting the BUG_ON() in
i915_gem_object_set_to_gpu_domain() as it gets called a second time
for the same object.
Signed-off-by: NKristian Høgsberg <krh@redhat.com>
Signed-off-by: NEric Anholt <eric@anholt.net>

This snuck in when I wrote phys object support.
Signed-off-by: NDave Airlie <airlied@redhat.com>

dev_priv->hw_status_page can be NULL, if i915_gem_retire_requests()
is called from i915_gem_busy_ioctl().

Signed-off-by Karsten Wiese <fzu@wemgehoertderstaat.de>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The object is dereferenced before the NULL check. Oops.

Fixes http://bugs.freedesktop.org/show_bug.cgi?id=20235Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This ensures that the user gets the latest information from the hardware
on whether the buffer is busy, potentially reducing the working set of objects
that the user chooses.
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@redhat.com>

In the KMS case, we need to suspend/resume GEM as well.  So on suspend, make
sure we idle GEM and stop any new rendering from coming in, and on resume,
re-init the framebuffer and clear the suspended flag.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@redhat.com>

The problem was that object_set_to_gpu_domain would set the new write_domains
that are getting set by this batchbuffer, then the accumulated flushes required
for all the objects in preparation for this batchbuffer were posted, and the
brand new write domain would get cleared by the flush being posted.  Instead,
hang on to the new (or old if we're not changing it) value and set it after
the flush is queued.

Results from this noticably included conformance test failures from reads
shortly after writes (where the new write domain had been lost and thus not
flushed and waited on), but is a suspected cause of hangs in some apps when
a write domain is lost on a buffer that gets reused for instruction or
commmand state.
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@redhat.com>

While not strictly required, it helped while thinking about the following
change.  This change should be invariant.
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This fixes potential fault at fault time if the object was unreferenced
while the mapping still existed.  Now, while the mmap_offset only lives
for the lifetime of the object, the object also stays alive while a vma
exists that needs it.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@redhat.com>

If we fail to create the ringbuffer, then we need to cleanup the allocated
hws.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

A missing unpin on the error path.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

A missing unpin on the error path.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

A missing unreference and unpin after rejecting the relocation for an
invalid memory domain.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

We failed to unlock the mutex after failing to create the mmap offset.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

Lockdep warns that i915_gem_execbuffer() can trigger a page fault (which
takes mmap_sem) while holding dev->struct_mutex, while drm_vm_open()
(which is called with mmap_sem already held) takes dev->struct_mutex.
So this is a potential AB-BA deadlock.

The way that i915_gem_execbuffer() triggers a page fault is by doing
copy_to_user() when returning new buffer offsets back to userspace;
however there is no reason to hold the struct_mutex when doing this
copy, since what is being copied is the contents of an array private to
i915_gem_execbuffer() anyway.  So we can fix the potential deadlock (and
get rid of the lockdep warning) by simply moving the copy_to_user()
outside of where struct_mutex is held.

This fixes <http://bugzilla.kernel.org/show_bug.cgi?id=12491>.
Reported-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Tested-by: NJesse Brandeburg <jesse.brandeburg@intel.com>
Signed-off-by: NRoland Dreier <rolandd@cisco.com>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

A missing unreference if the user calls pin() a second time on a pinned
buffer.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

Also spotted by Owain Ainsworth.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

The DRI people seem to have a hard time getting these right (see also
commit aeb565df).
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

If we failed to allocate a new fence register we would return
VM_FAULT_SIGBUS without relinquishing the lock.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

Adds code to set up fence registers at execbuf time on pre-965 chips as
necessary.  Also fixes up a few bugs in the pre-965 tile register support
(get_order != ffs).  The number of fences available to the kernel defaults
to the hw limit minus 3 (for legacy X front/back/depth), but a new parameter
allows userspace to override that as needed.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

Previously, the caller would continue along without knowing that the
function failed, resulting in potential mis-rendering.  Right now vm_fault
just returns SIGBUS in that case, and we may need to disable signal handling
to avoid that happening.
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

We'd love to just be using PAT, but even on chips with PAT it gets disabled
sometimes due to an errata.  It would probably be better to have pat_enabled
exported and only bother with this when !pat_enabled.
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

Fixes hangs when starting X for the second time.
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

Use '%zu' to print out a size_t variable, not '%d'.  Another case of the
"let's keep at least Linus' defconfig compile warningless" rule.
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This off-by-one was pointed out by Jesse Barnes.
Signed-off-by: NDave Airlie <airlied@redhat.com>

This is an initial patch to do support for objects which needs physical
contiguous main ram, cursors and overlay registers on older chipsets.

These objects are bound on cursor bin, like pinning, and we copy
the data to/from the backing store object into the real one on attach/detach.

notes:
possible over the top in attach/detach operations.
no overlay support yet.
Signed-off-by: NDave Airlie <airlied@redhat.com>

__FUNCTION__ is gcc-specific, use __func__
Signed-off-by: NHarvey Harrison <harvey.harrison@gmail.com>
Acked-by: NMauro Carvalho Chehab <mchehab@redhat.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This avoids a BUG_ON in the enter_vt path due to objects being in the GTT
when we shouldn't have ever let them be (as we're not supposed to touch the
device during that time).

This was triggered by a change in the 2D driver to use the GTT mapping of
objects after pinning them to improve software fallback performance.
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

The error path for object list being null is in the second goto target.
Signed-off-by: NJulia Lawall <julia@diku.dk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

This showed up in logs where people had a hung chip, so pinning was blocked
on the chip unpinning other buffers, and the X Server took its scheduler
signal during that time.
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>

We haven't seen this in practice, but it was visible when looking at a bug
report from when i915_gem_evict_everything() was broken and would always
return error.
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NDave Airlie <airlied@linux.ie>