The recursion loop goes retire_requests->unbind->gpu_idle->retire_reqeusts.

Every time we go through this we need a
- active object that can be retired
- and there are no other references to that object than the one from
  the active list, so that it gets unbound and freed immediately.
Otherwise the recursion stops. So the recursion is only limited by the
number of objects that fit these requirements sitting in the active list
any time retire_request is called.

Issue exercised by tests/gem_unref_active_buffers from i-g-t.

There's been a decent bikeshed discussion whether it wouldn't be
better to pass around a flag, but imo this is o.k. for such a limited
case that only supports a w/a.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=42180Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NChris Wilson <chris@chris-wilson>
[ickle- we built better bikesheds, but this keeps the rain off for now]
Tested-by: NDave Airlie <airlied@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

A call to i915_add_request() has been made in function i915_gem_busy_ioctl(). i915_add_request can fail,
so in it's exit path previously allocated memory needs to be freed.
Signed-off-by: NRakib Mullick <rakib.mullick@gmail.com>
Reviewed-by: NKeith Packard <keithp@keithp.com>
Signed-off-by: NKeith Packard <keithp@keithp.com>

I've been seeing memory leaks on my system in the form of large
(300-400MB) GEM objects created by now-dead processes laying around
clogging up memory.  I usually notice when it gets to about 1.2GB of
them.  Hopefully this clears up the issue, but I just found this bug
by inspection.
Signed-off-by: NEric Anholt <eric@anholt.net>
Cc: stable@kernel.org
Signed-off-by: NKeith Packard <keithp@keithp.com>

IVB supports these bits as well.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NKeith Packard <keithp@keithp.com>

In preparation of to support 32 fences on Ivybdrigde.
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NKeith Packard <keithp@keithp.com>

I've been seeing memory leaks on my system in the form of large
(300-400MB) GEM objects created by now-dead processes laying around
clogging up memory.  I usually notice when it gets to about 1.2GB of
them.  Hopefully this clears up the issue, but I just found this bug
by inspection.
Signed-off-by: NEric Anholt <eric@anholt.net>
Cc: stable@kernel.org
Signed-off-by: NKeith Packard <keithp@keithp.com>

[Description from: Daniel Vetter]
I've just discussed this quickly with Chris on irc and it's probably
best to just kill the list_empty early bailout. gpu_idle isn't a
fastpath, so who cares. One candidate where we emit commands to the ring
without adding anything onto these lists is e.g. pageflip. There are
probably more.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NKeith Packard <keithp@keithp.com>

We currently only round up the userspace size to the next page. We
assume that userspace hasn't made a mistake and requested a zero-length
gem object and all through our internal code we then presume that every
object is backed by at least a single page. Fix that oversight and
report EINVAL back to userspace if they try to create a zero length
object.

[danvet: This fixes tests/gem_bad_length]
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Use the helper function already employed by the pwrite/pread
functions.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Various issues involved with the space character were generating
warnings in the checkpatch.pl file. This patch removes most of those
warnings.
Signed-off-by: NAkshay Joshi <me@akshayjoshi.com>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Signed-off-by: NRob Clark <rob@ti.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Failing to pin a scanout buffer will most likely lead to a black
screen, so if the GPU is wedged, then just let the pin happen and hope
that things work out OK.
Signed-off-by: NKeith Packard <keithp@keithp.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Failing to pin a scanout buffer will most likely lead to a black
screen, so if the GPU is wedged, then just let the pin happen and hope
that things work out OK.

v2: Just ignore any error from i915_gem_object_wait_rendering, as
suggested by Chris Wilson
Signed-off-by: NKeith Packard <keithp@keithp.com>

Align unfenced buffers on older hardware to the power-of-two object
size.  The docs suggest that it should be possible to align only to a
power-of-two tile height, but using the already computed fence size is
easier and always correct. We also have to make sure that we unbind
misaligned buffers upon tiling changes.

In order to prevent a repetition of this bug, we change the interface
to the alignment computation routines to force the caller to provide
the requested alignment and size of the GTT binding rather than assume
the current values on the object.
Reported-and-tested-by: NSitosfe Wheeler <sitsofe@yahoo.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=36326Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: stable@kernel.org
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Provide a parameter to disable hanghcheck. This is useful mostly for
developers trying to debug known problems, and probably should not be
touched by normal users.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Konstantin Belousov pointed out that 4697995b replaced the generic
i915_driver_irq_*install() functions with chipset specific routines
accessible only through driver->irq_*install(). So update the sanity
check in i915_request_wait() to match.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NKeith Packard <keithp@keithp.com>

The interface to ->truncate_range is changing very slightly: once "tmpfs:
take control of its truncate_range" has been applied, this can be applied.
 For now there is only a slight inefficiency while this remains unapplied,
but it will soon become essential for managing shmem's use of swap.

Change i915_gem_object_truncate() to use shmem_truncate_range() directly:
which should also spare i915 later change if we switch from
inode_operations->truncate_range to file_operations->fallocate.
Signed-off-by: NHugh Dickins <hughd@google.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Keith Packard <keithp@keithp.com>
Cc: Dave Airlie <airlied@redhat.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Soon tmpfs will stop supporting ->readpage and read_cache_page_gfp(): once
"tmpfs: add shmem_read_mapping_page_gfp" has been applied, this patch can
be applied to ease the transition.

Make i915_gem_object_get_pages_gtt() use shmem_read_mapping_page_gfp() in
the one place it's needed; elsewhere use shmem_read_mapping_page(), with
the mapping's gfp_mask properly initialized.

Forget about __GFP_COLD: since tmpfs initializes its pages with memset,
asking for a cold page is counter-productive.

Include linux/shmem_fs.h also in drm_gem.c: with shmem_file_setup() now
declared there too, we shall remove the prototype from linux/mm.h later.
Signed-off-by: NHugh Dickins <hughd@google.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Keith Packard <keithp@keithp.com>
Cc: Dave Airlie <airlied@redhat.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Even if the object is no longer in the GTT domain, there may still be
a user space mapping which needs to be released.

Without this fix, render-based text (mostly in firefox) would
occasionally get corrupted when the system was under load.
Signed-off-by: NKeith Packard <keithp@keithp.com>

This reverts commit 4a684a41.
Userland has always been required to set the object's domain to GTT
before using it through a GTT mapping, it's not something that the
kernel is supposed to enforce.  (The pagefault support is so that we
can handle multiple mappings without userland having to pin across
them, not so that userland can use GTT after GPU domains without
telling the kernel).

Fixes 19.2% +/- 0.8% (n=6) performance regression in cairo-gl
firefox-talos-gfx on my T420 latop.
Signed-off-by: NKeith Packard <keithp@keithp.com>

It seems to me that we are leaking 'user_pages' in
drivers/gpu/drm/i915/i915_gem.c::i915_gem_shmem_pread_slow() if
read_cache_page_gfp() fails.
Signed-off-by: NJesper Juhl <jj@chaosbits.net>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Improves full-screen openarena on my laptop 20.3% +/- 4.0% (n=3)
Improves 800x600 nexuiz on my laptop 12.3% +/- 0.1% (n=3)

We have more room to improve with doing LLC caching for display using
GFDT, and in doing LLC+MLC caching, but this was an easy performance
win and incremental improvement toward those two.
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The simplest and common method for ensuring scanout coherency on all
chipsets is to mark the scanout buffers as uncached (and for
userspace to remember to flush the render cache every so often).

We can improve upon this for later generations by marking scanout
objects as GFDT and only flush those cachelines when required. However,
we start simple.

[v2: Move the set to uncached above the clflush.  Otherwise, we'd skip
the clflush and try to scan out data that was still sitting in the
cache.]
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We need to perform a few operations in order to move the object into the
display plane (where it can be accessed coherently by the display
engine) that are important for future safety to forbid whilst pinned. As a
result, we want to need to perform some of the operations before pinning,
but some are required once we have been bound into the GTT. So combine
the pinning performed by all the callers with set_to_display_plane(), so
this complication is contained within the single function.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

[anholt v2: Don't forget that when going from cached to uncached, we
haven't been tracking the write domain from the CPU perspective, since
we haven't needed it for GPU coherency.]

[ickle v3: We also need to make sure we relinquish any fences on older
chipsets and clear the GTT for sane domain tracking.]
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Like its siblings finish_gpu(), this function clears the object from the
GTT domain forcing it to be trigger a domain invalidation should we ever
need to use via the GTT again.

Note that the most important side-effect of finishing the GTT domain
(aside from clearing the tracking read/write domains) is that it imposes
an memory barrier so that all accesses are complete before it returns,
which is important if you intend to be modifying translation tables
shortly afterwards. The second most important side-effect is that it
tears down the GTT mappings forcing a page-fault and invalidation on
next user access to the object.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

... reincarnated from i915_gem_object_flush_gpu(). The semantic
difference is that after calling finish_gpu() the object no longer
resides in any GPU domain, and so will cause the GPU caches to be
invalidated if it is ever used again.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

A tile on gen2 has a size of 2kb, stride of 128 bytes and 16 rows.

Userspace was broken and assumed 8 rows. Chris Wilson noted that the
kernel unfortunately can't reliable check that because libdrm rounds
up the size to the next bucket.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Convert our open coded offset_in_page() to the common macro.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NKeith Packard <keithp@keithp.com>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Change each shrinker's API by consolidating the existing parameters into
shrink_control struct.  This will simplify any further features added w/o
touching each file of shrinker.

[akpm@linux-foundation.org: fix build]
[akpm@linux-foundation.org: fix warning]
[kosaki.motohiro@jp.fujitsu.com: fix up new shrinker API]
[akpm@linux-foundation.org: fix xfs warning]
[akpm@linux-foundation.org: update gfs2]
Signed-off-by: NYing Han <yinghan@google.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
Acked-by: NPavel Emelyanov <xemul@openvz.org>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Acked-by: NRik van Riel <riel@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Hugh Dickins <hughd@google.com>
Cc: Dave Hansen <dave@linux.vnet.ibm.com>
Cc: Steven Whitehouse <swhiteho@redhat.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The registers are the same as on Sandybridge.  Fixes scrambled display
in X when it does software drawing to the GTT, and scans the results
out as tiled.
Signed-off-by: NEric Anholt <eric@anholt.net>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NKeith Packard <keithp@keithp.com>

This is once less place to miss a new INTEL_INFO(dev)->gen update now.
Signed-off-by: NEric Anholt <eric@anholt.net>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Rely on the GPU snooping into the CPU cache for appropriately bound
objects on MI_FLUSH. Or perhaps one day we will have a cache-coherent
CPU/GPU package...
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NKeith Packard <keithp@keithp.com>

... to clarify just how we use it inside the driver and remove the
confusion of the poorly matching agp_type names. We still need to
translate through agp_type for interface into the fake AGP driver.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Found by gem_stress.

As we perform retirement from a workqueue, it is possible for us to free
and unbind objects after the last close on the device, and so after the
address space has been torn down and reset to NULL:

BUG: unable to handle kernel NULL pointer dereference at 00000054
IP: [<c1295a20>] mutex_lock+0xf/0x27
*pde = 00000000
Oops: 0002 [#1] SMP
last sysfs file: /sys/module/vt/parameters/default_utf8

Pid: 5, comm: kworker/u:0 Not tainted 2.6.38+ #214
EIP: 0060:[<c1295a20>] EFLAGS: 00010206 CPU: 1
EIP is at mutex_lock+0xf/0x27
EAX: 00000054 EBX: 00000054 ECX: 00000000 EDX: 00012fff
ESI: 00000028 EDI: 00000000 EBP: f706fe20 ESP: f706fe18
 DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
Process kworker/u:0 (pid: 5, ti=f706e000 task=f7060d00 task.ti=f706e000)
Stack:
 f5aa3c60 00000000 f706fe74 c107e7df 00000246 dea55380 00000054 f5aa3c60
 f706fe44 00000061 f70b4000 c13fff84 00000008 f706fe54 00000000 00000000
 00012f00 00012fff 00000028 c109e575 f6b36700 00100000 00000000 f706fe90
Call Trace:
 [<c107e7df>] unmap_mapping_range+0x7d/0x1e6
 [<c109e575>] ? mntput_no_expire+0x52/0xb6
 [<c11c12f6>] i915_gem_release_mmap+0x49/0x58
 [<c11c3449>] i915_gem_object_unbind+0x4c/0x125
 [<c11c353f>] i915_gem_free_object_tail+0x1d/0xdb
 [<c11c55a2>] i915_gem_free_object+0x3d/0x41
 [<c11a6be2>] ? drm_gem_object_free+0x0/0x27
 [<c11a6c07>] drm_gem_object_free+0x25/0x27
 [<c113c3ca>] kref_put+0x39/0x42
 [<c11c0a59>] drm_gem_object_unreference+0x16/0x18
 [<c11c0b15>] i915_gem_object_move_to_inactive+0xba/0xbe
 [<c11c0c87>] i915_gem_retire_requests_ring+0x16e/0x1a5
 [<c11c3645>] i915_gem_retire_requests+0x48/0x63
 [<c11c36ac>] i915_gem_retire_work_handler+0x4c/0x117
 [<c10385d1>] process_one_work+0x140/0x21b
 [<c103734c>] ? __need_more_worker+0x13/0x2a
 [<c10373b1>] ? need_to_create_worker+0x1c/0x35
 [<c11c3660>] ? i915_gem_retire_work_handler+0x0/0x117
 [<c1038faf>] worker_thread+0xd4/0x14b
 [<c1038edb>] ? worker_thread+0x0/0x14b
 [<c103be1b>] kthread+0x68/0x6d
 [<c103bdb3>] ? kthread+0x0/0x6d
 [<c12970f6>] kernel_thread_helper+0x6/0x10
Code: 00 e8 98 fe ff ff 5d c3 55 89 e5 3e 8d 74 26 00 ba 01 00 00 00 e8
84 fe ff ff 5d c3 55 89 e5 53 8d 64 24 fc 3e 8d 74 26 00 89 c3 <f0> ff
08 79 05 e8 ab ff ff ff 89 e0 25 00 e0 ff ff 89 43 10 58
EIP: [<c1295a20>] mutex_lock+0xf/0x27 SS:ESP 0068:f706fe18
CR2: 0000000000000054
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NKeith Packard <keithp@keithp.com>

Detected by scripts/coccinelle/free/kfree.cocci.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NKeith Packard <keithp@keithp.com>

We always skipped flushing the BLT ring if the request flush did not
include the RENDER domain. However, this neglects that we try to flush
the COMMAND domain after every batch and before the breadcrumb interrupt
(to make sure the batch is indeed completed prior to the interrupt
firing and so insuring CPU coherency). As a result of the missing flush,
incoherency did indeed creep in, most notable when using lots of command
buffers and so potentially rewritting an active command buffer (i.e.
the GPU was still executing from it even though the following interrupt
had already fired and the request/buffer retired).

As all ring->flush routines now have the same preconditions, de-duplicate
and move those checks up into i915_gem_flush_ring().

Fixes gem_linear_blit.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=35284Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Tested-by: mengmeng.meng@intel.com

Cc: Dave Airlie <airlied@linux.ie>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

... even though it was disabled. A mistake in the handling of fence reuse
caused us to skip the vital delay of waiting for the object to finish
rendering before changing the register. This resulted in us changing the
fence register whilst the bo was active and so causing the blits to
complete using the wrong stride or even the wrong tiling. (Visually the
effect is that small blocks of the screen look like they have been
interlaced). The fix is to wait for the GPU to finish using the memory
region pointed to by the fence before changing it.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=34584
Cc: Andy Whitcroft <apw@canonical.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
[Note for 2.6.38-stable, we need to reintroduce the interruptible passing]
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Tested-by: NDave Airlie <airlied@linux.ie>