... and a few other defines.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Currently if we hit a pagefault when applying a user relocation for the
execbuffer, we bail and return EFAULT to the application. Instead, we
need to unwind, drop the dev->struct_mutex, copy all the relocation
entries to a vmalloc array (to avoid any potential circular deadlocks
when resolving the pagefault), retake the mutex and then apply the
relocations.  Afterwards, we need to again drop the lock and copy the
vmalloc array back to userspace.

v2: Incorporate feedback from Daniel Vetter.
Reported-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Commit 2549d6c2 removed the vmalloc used for temporary storage of the
relocation lists used during execbuffer. However, our use of vmalloc was
being protected by an integer overflow check which we do want to
preserve!
Reported-by: NDan Carpenter <error27@gmail.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Linus Torvalds found that it was rather trivial to trigger a system
freeze:

  In fact, with lockdep, I don't even need to do the sysrq-d thing: it
  shows the bug as it happens. It's the X server taking the same lock
  recursively.

  Here's the problem:

    =============================================
    [ INFO: possible recursive locking detected ]
    2.6.37-rc2-00012-gbdbd01ac #7
    ---------------------------------------------
    Xorg/2816 is trying to acquire lock:
     (&dev->struct_mutex){+.+.+.}, at: [<ffffffff812c626c>] i915_gem_fault+0x50/0x17e

    but task is already holding lock:
     (&dev->struct_mutex){+.+.+.}, at: [<ffffffff812c403b>] i915_mutex_lock_interruptible+0x28/0x4a

    other info that might help us debug this:
    2 locks held by Xorg/2816:
     #0:  (&dev->struct_mutex){+.+.+.}, at: [<ffffffff812c403b>] i915_mutex_lock_interruptible+0x28/0x4a
     #1:  (&mm->mmap_sem){++++++}, at: [<ffffffff81022d4f>] page_fault+0x156/0x37b

This recursion was introduced by rearranging the locking to avoid the
double locking on the fast path (4f27b5d and fbd5a26d) and the
introduction of the prefault to encourage the fast paths (b5e4f2b). In
order to undo the problem, we rearrange the code to perform the access
validation upfront, attempt to prefault and then fight for control of the
mutex.  the best case scenario where the mutex is uncontended the
prefaulting is not wasted.
Reported-and-tested-by: NLinus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

g33/pineview doesn't have any alignment constrains for unfenced tiled
buffers. But older chips have. Fix this.

Problem introduced in a00b10c3.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

An old and oft reported bug, is that of the GPU hanging on a
MI_WAIT_FOR_EVENT following a mode switch. The cause is that the GPU is
waiting on a scanline counter on an inactive pipe, and so waits for a
very long time until eventually the user reboots his machine.

We can prevent this either by moving the WAIT into the kernel and
thereby incurring considerable cost on every swapbuffers, or by waiting
for the GPU to retire the last batch that accesses the framebuffer
before installing a new one. As mode switches are much rarer than swap
buffers, this looks like an easy choice.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=28964
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=29252Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: stable@kernel.org

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Coalesce long formats.
Align arguments.
Add missing newlines.
Signed-off-by: NJoe Perches <joe@perches.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

... and so prevent a potential circular reference:

  [ INFO: possible circular locking dependency detected ]
  2.6.37-rc1-uwe1+ #4
  -------------------------------------------------------
  Xorg/1401 is trying to acquire lock:
   (&mm->mmap_sem){++++++}, at: [<c01e4ddb>] might_fault+0x4b/0xa0

  but task is already holding lock:
   (&dev->struct_mutex){+.+.+.}, at: [<f869c3ac>]
  i915_mutex_lock_interruptible+0x3c/0x60 [i915]

  which lock already depends on the new lock.

When the locking around the pwrite ioctl was simplified, I did not spot
that the phys path never took any locks and so we introduced this
potential circular reference.
Reported-by: NUwe Helm <uwe.helm@googlemail.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Instead of killing the process, just return no page found and reschedule
the process giving the GPU some time to (hopefully) recover.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

a00b10c3 "Only enforce fence limits inside the GTT" also
added a fenceable/mappable disdinction when binding/pinning buffers.
This only complicates the code with no pratical gain:

- In execbuffer this matters on for g33/pineview, as this is the only
  chip that needs fences and has an unmappable gtt area. But fences
  are only possible in the mappable part of the gtt, so need_fence
  implies need_mappable. And need_mappable is only set independantly
  with relocations which implies (for sane userspace) that the buffer
  is untiled.

- The overlay code is only really used on i8xx, which doesn't have
  unmappable gtt. And it doesn't support tiled buffers, currently.

- For all other buffers it's a bug to pass in a tiled bo.

In short, this disdinction doesn't have any practical gain.

I've also reverted mapping the overlay and context pages as possibly
unmappable. It's not worth being overtly clever here, all the big
gains from unmappable are for execbuf bos.

Also add a comment for a clever optimization that confused me
while reading the original patch by Chris Wilson.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

When merging Daniel's full-gtt patches I had a set of tweaks which I
thought I had undone. I was half right...

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=31286
Reported-by: jinjin.wang@intel.com
Reported-by: NAlexey Fisher <bug-track@fisher-privat.net>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: stable@kernel.org

... and into a local structure scoped for the single function in which
it is used.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Take two passes to evict everything whilst searching for sufficient free
space to bind the batchbuffer. After searching for sufficient free space
using LRU eviction, evict everything that is purgeable and try again.
Only then if there is insufficient free space (or the GTT is too badly
fragmented) evict everything from the aperture and try one last time.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Accessing the uninitialised obj->pages instead of the local page lead to
an OOPs.
Reported-by: NXavier Chantry <chantry.xavier@gmail.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

So long as we adhere to the fence registers rules for alignment and no
overlaps (including with unfenced accesses to linear memory) and account
for the tiled access in our size allocation, we do not have to allocate
the full fenced region for the object. This allows us to fight the bloat
tiling imposed on pre-i965 chipsets and frees up RAM for real use. [Inside
the GTT we still suffer the additional alignment constraints, so it doesn't
magic allow us to render larger scenes without stalls -- we need the
expanded GTT and fence pipelining to overcome those...]
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Also spotted by Dan Carpenter.

obj->pin_count is unsigned so the BUG_ON(obj->pin_count<0) will never
trigger.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

The error code is only expected during the actual pruning and not during
the first measurement (nr_to_scan == 0) pass.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

It is possible for the active list to only contain a read-only buffer so
that the ring->gpu_write_list remains entry. This leads to an
inconsistency between i915_gpu_is_active() and i915_gpu_idle() causing
an infinite spin during the shrinker and an assertion failure that
i915_gpu_idle() does indeed flush all buffers from the active lists.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

In order to force a page-fault on a GTT mapping after we start using it
from the GPU and so enforce correct CPU/GPU synchronisation, we need to
invalidate the mapping.

Pointed out by Owain G. Ainsworth.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

By using read_cache_page() for individual pages during pwrite/pread we
can eliminate an unnecessary large allocation (and immediate free) of
obj->pages. Also this eliminates any potential nesting of get/put pages,
simplifying the code and preparing the path for greater things.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Since we rarely use the mmap_offset and it is easily computable from the
obj->map_list.hash, remove it.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Eliminate the racy device unload by embedding a shrinker into each
device. Smaller, simpler code.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

More precisely: For those that _need_ to be mappable. Also add two
BUG_ONs in fault and pin to check the consistency of the mappable
flag.

Changes in v2:
- Add tracking of gtt mappable space (to notice mappable/unmappable
  balancing issues).
- Improve the mappable working set tracking by tracking fault and pin
  separately.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

This way we can make some more educated guesses as to why exactly
we can't use 2G apertures to their full potential ;)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

At least the part that's currently enabled by the BIOS.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

In i915_gem_object_pin obviously unbind only if mappable is true.

This is the last part to enable gtt_mappable_end != gtt_size, which
the next patch will do.

v2: Fences on g33/pineview only work in the mappable part of the
gtt.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Like before add a parameter mappable (also to gem_object_pin) and
set it depending upon the context. Only bos that are brought into
the gtt due to an execbuffer call can be put into the unmappable
part of the gtt, everything else (especially pinned objects) need
to be put into the mappable part of the gtt.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Add a mappable parameter to i915_gem_evict_something to distinguish
the two cases (non-restricted vs. mappable gtt allocations). No
functional changes because the mappable limit is set to the end of
the gtt currently.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

In the event that we fail to dispatch the execbuffer, for example if
there is insufficient space on the ring, we were leaving the objects in
an inconsistent state. Notably they were marked as being in the GPU
write domain, but were not added to the ring or any list. This would
lead to inevitable oops:

[ 1010.522940] [drm:i915_gem_do_execbuffer] *ERROR* dispatch failed -16
[ 1010.523055] BUG: unable to handle kernel NULL pointer dereference at
0000000000000088
[ 1010.523097] IP: [<ffffffff8122d006>] i915_gem_flush_ring+0x26/0x140
[ 1010.523120] PGD 14cf2f067 PUD 14ce04067 PMD 0
[ 1010.523140] Oops: 0000 [#1] SMP
[ 1010.523154] last sysfs file: /sys/devices/virtual/vc/vcsa2/uevent
[ 1010.523173] CPU 0
[ 1010.523183] Pid: 716, comm: X Not tainted 2.6.36+ #34 LosLunas
CRB/SandyBridge Platform
[ 1010.523206] RIP: 0010:[<ffffffff8122d006>]  [<ffffffff8122d006>]
i915_gem_flush_ring+0x26/0x140
[ 1010.523233] RSP: 0018:ffff88014bf97cd8  EFLAGS: 00010296
[ 1010.523249] RAX: ffff88014e2d1808 RBX: 0000000000000000 RCX: 0000000000000000
[ 1010.523270] RDX: 0000000000000002 RSI: 0000000000000000 RDI: 0000000000000000
[ 1010.523290] RBP: ffff88014e2d1000 R08: 0000000000000002 R09: 00000000400c645f
[ 1010.523311] R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000002
[ 1010.523331] R13: ffff88014e29a000 R14: 00000000000000c8 R15: ffffffff8162eb28
[ 1010.523352] FS:  00007fc62379d700(0000) GS:ffff88001fc00000(0000) knlGS:0000000000000000
[ 1010.523375] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1010.523392] CR2: 0000000000000088 CR3: 000000014bf87000 CR4: 00000000000406f0
[ 1010.523412] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1010.523433] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 1010.523454] Process X (pid: 716, threadinfo ffff88014bf96000, task ffff88014cc1ee40)
[ 1010.523475] Stack:
[ 1010.523483]  ffff88014d5199c0 0000000000000200 0000000000000000 ffff88014bcc6400
[ 1010.523509] <0> 0000000000000000 0000000000000001 ffff88014e29a000 ffff88014bcc6400
[ 1010.523537] <0> ffffffff8162eb28 ffffffff8122faa8 ffff88014e29a000 ffff88014bcc6400
[ 1010.523568] Call Trace:
[ 1010.523578]  [<ffffffff8122faa8>] ?  i915_gem_object_flush_gpu_write_domain+0x48/0x80
[ 1010.523601]  [<ffffffff8122fb8e>] ?  i915_gem_object_set_to_gtt_domain+0x2e/0xb0
[ 1010.523623]  [<ffffffff8123113b>] ?  i915_gem_set_domain_ioctl+0xdb/0x1f0
[ 1010.523644]  [<ffffffff8120a3f1>] ? drm_ioctl+0x3d1/0x460
[ 1010.523660]  [<ffffffff81231060>] ?  i915_gem_set_domain_ioctl+0x0/0x1f0
[ 1010.523682]  [<ffffffff81092618>] ?  vma_prio_tree_insert+0x28/0x120
[ 1010.523701]  [<ffffffff8109f379>] ? vma_link+0x99/0xf0
[ 1010.523717]  [<ffffffff810a111d>] ? mmap_region+0x1ed/0x4f0
[ 1010.523734]  [<ffffffff810c306f>] ? do_vfs_ioctl+0x9f/0x580
[ 1010.523750]  [<ffffffff810c3599>] ? sys_ioctl+0x49/0x80
[ 1010.523767]  [<ffffffff810022eb>] ?  system_call_fastpath+0x16/0x1b
[ 1010.523785] Code: 00 00 00 00 00 41 57 89 ce 41 56 41 55 41 54 45 89 c4 55 48 89 fd 53 48 89 d3 44 89 c2 48 89 df 4c 8d b3 c8 00 00 00 48 83 ec 18 <ff> 93 88 00 00 00 48 8b 83 c8 00 00 00 4c 8b bd 30 03 00 00 48
[ 1010.523946] RIP  [<ffffffff8122d006>] i915_gem_flush_ring+0x26/0x140
[ 1010.523966]  RSP <ffff88014bf97cd8>
[ 1010.523977] CR2: 0000000000000088
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Preparing the ringbuffer for adding new commands can fail (a timeout
whilst waiting for the GPU to catch up and free some space). So check
for any potential error before overwriting HEAD with new commands, and
propagate that error back to the user where possible.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

The ringbuffer keeps a pointer to the parent device, so we can use that
instead of passing around the pointer on the stack.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Keep the current interface but ignore the KM_type and use a stack based
approach.

The advantage is that we get rid of crappy code like:

	#define __KM_PTE			\
		(in_nmi() ? KM_NMI_PTE : 	\
		 in_irq() ? KM_IRQ_PTE :	\
		 KM_PTE0)

and in general can stop worrying about what context we're in and what kmap
slots might be appropriate for that.

The downside is that FRV kmap_atomic() gets more expensive.

For now we use a CPP trick suggested by Andrew:

  #define kmap_atomic(page, args...) __kmap_atomic(page)

to avoid having to touch all kmap_atomic() users in a single patch.

[ not compiled on:
  - mn10300: the arch doesn't actually build with highmem to begin with ]

[akpm@linux-foundation.org: coding-style fixes]
[akpm@linux-foundation.org: fix up drivers/gpu/drm/i915/intel_overlay.c]
Acked-by: NRik van Riel <riel@redhat.com>
Signed-off-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
Acked-by: NChris Metcalf <cmetcalf@tilera.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: David Miller <davem@davemloft.net>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Dave Airlie <airlied@linux.ie>
Cc: Li Zefan <lizf@cn.fujitsu.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

... to prevent flush processing of an idle (or even absent) ring.

This fixes a regression during suspend from 87acb0a5.
Reported-and-tested-by: NAlexey Fisher <bug-track@fisher-privat.net>
Tested-by: NPeter Clifton <pcjc2@cam.ac.uk>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>