We have it, so use it. This required moving the function to avoid
a forward declaration.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NEric Anholt <eric@anholt.net>

The fence_list should be lru ordered for otherwise we might try
to steal a fence reg from an active object even though there are
fences from inactive objects available. lru ordering was obeyed
for gpu access everywhere save when moving dirty objects from
flushing_list to active_list.

Fixing this cause the code to indent way to much, so I've extracted
the flushing_list processing logic into its on function.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

The spaghetti logic in there tripped up my brain's code parser for a
few secs. Prevent this from happening again by extracting the fence
stealing code into a seperate functions. IMHO this slightly clears up
the code flow.

v2: Beautified according to ickle's comments.
v3: ickle forgot to flush his comment queue ... Now there's also a
we-are-paranoid BUG_ON in there.
v4: I've forgotten to switch on my brain when doing v3. Now the BUG_ON
actually checks something useful.
v5: Clean up a stale comment as noted by Eric Anholt.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

All other accesses take this spinlock, so do this here, too.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NEric Anholt <eric@anholt.net>

This has a few functional changes against the old code:

* a few more unnecessary loads and stores to the drm_i915_fence_reg
  objects. Also an unnecessary store to the hw fence register.

* zaps any userspace mappings before doing other flushes. Only changes
  anything when userspace does racy stuff against itself.

* also flush GTT domain. This is a noop, but still try to keep the
  bookkeeping correct.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NEric Anholt <eric@anholt.net>

The register's moved to the same location as the one for the BCS, it seems.
Signed-off-by: NEric Anholt <eric@anholt.net>

Signed-off-by: NEric Anholt <eric@anholt.net>

Signed-off-by: NEric Anholt <eric@anholt.net>

i915_gem_object_fenceable was mostly just a repeat of the
i915_gem_object_fence_offset_ok, but also checking the size (which was
checkecd when we allowed that BO to be tiled in the first place). So
instead, export the latter function and use it in place.
Signed-Off-By: NOwain G. Ainsworth <oga@openbsd.org>
Signed-off-by: NEric Anholt <eric@anholt.net>

This aligns it with the other user of i915_gem_clear_fence_reg,
which blows away the mapping before changing the fence reg.

Only affects userspace if it races against itself when changing
tiling parameters, i.e. behaviour is undefined, anyway.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

No functional change, because gtt flushing is a no-op. Still, try
to keep the bookkeeping accurate. The if is still slightly wrong
for with execbuf2 even i915-class hw doesn't always need a fence
reg for gpu access. But that's for somewhen lateron.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NEric Anholt <eric@anholt.net>

The fence start is for compatibility with UMS X Servers before fence
management.  KMS X Servers only started doing tiling after fence
management appeared.
Signed-off-by: NEric Anholt <eric@anholt.net>

With the introduction of the hang-check, we can safely expect that
i915_wait_request() will always return even when the GPU hangs, and so
do not need to open code the wait in order to manually check for the
hang. Also we do not need to always evict all buffers, so only flush
the GPU (and wait for it to idle) for KMS, but continue to evict for UMS.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

Mostly obvious simplifications.

The i915 pread/pwrite ioctls, intel_overlay_put_image and
nouveau_gem_new were incorrectly using the locked versions
without locking: this is also fixed in this patch.
Signed-off-by: NLuca Barbieri <luca@luca-barbieri.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NOwain G. Ainsworth <oga@openbsd.org>
Signed-off-by: NEric Anholt <eric@anholt.net>

Before changing the status of a buffer with a pending write we will await
upon a new flush for that buffer. So we can take advantage of any flushes
posted whilst the buffer is active and pending processing by the GPU, by
clearing its write_domain and updating its last_rendering_seqno -- thus
saving a potential flush in deep queues and improves flushing behaviour
upon eviction for both GTT space and fences.

In order to reduce the time spent searching the active list for matching
write_domains, we move those to a separate list whose elements are
the buffers belong to the active/flushing list with pending writes.

Orignal patch by Chris Wilson <chris@chris-wilson.co.uk>, forward-ported
by me.

In addition to better performance, this also fixes a real bug. Before
this changes, i915_gem_evict_everything didn't work as advertised. When
the gpu was actually busy and processing request, the flush and subsequent
wait would not move active and dirty buffers to the inactive list, but
just to the flushing list. Which triggered the BUG_ON at the end of this
function. With the more tight dirty buffer tracking, all currently busy and
dirty buffers get moved to the inactive list by one i915_gem_flush operation.

I've left the BUG_ON I've used to prove this in there.

References:
  Bug 25911 - 2.10.0 causes kernel oops and system hangs
  http://bugs.freedesktop.org/show_bug.cgi?id=25911

  Bug 26101 - [i915] xf86-video-intel 2.10.0 (and git) triggers kernel oops
              within seconds after login
  http://bugs.freedesktop.org/show_bug.cgi?id=26101Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Tested-by: NAdam Lantos <hege@playma.org>
Cc: stable@kernel.org
Signed-off-by: NEric Anholt <eric@anholt.net>

Following a gpu hang, we would leak the relocation buffer. So simply
earrange the error path to always free the relocation buffer.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

Having missed the ENOMEM return via i915_gem_fault(), there are probably
other paths that I also missed. By not enabling NORETRY by default these
paths can run the shrinker and take memory from the system (but not from
our own inactive lists because our shrinker can not run whilst we hold
the struct mutex) and this may allow the system to survive a little longer
whilst our drivers consume all available memory.

References:
  OOM killer unexpectedly called with kernel 2.6.32
  http://bugzilla.kernel.org/show_bug.cgi?id=14933

v2: Pass gfp into page mapping.
v3: Use new read_cache_page_gfp() instead of open-coding.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Hugh Dickins <hugh.dickins@tiscali.co.uk>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Cc: Eric Anholt <eric@anholt.net>
Cc: stable@kernel.org
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

X.org hang with [drm:i915_gem_do_execbuffer] *ERROR* in dmesg
  http://bugzilla.kernel.org/show_bug.cgi?id=15114

Matej found he was hitting an error path within i915_gem_do_execbuffer()
that led to the attempt to dereference an uninitialised pointer during
cleanup. This path used to be safe as we used to calloc the object
lists, but this was changed in c8e0f93a. Daniel Vetter had also spotted
this error and proposed a similar patch.

[ 6379.732892] [drm:i915_gem_do_execbuffer] *ERROR* Object ffff880098cd6540 appears more than once in object list
[ 6379.740976] [drm:i915_gem_do_execbuffer] *ERROR* Object ffff880098cd6540 appears more than once in object list
[ 6379.740995] BUG: unable to handle kernel NULL pointer dereference at 00000000000000a0
[ 6379.740998] IP: [<ffffffff8122ddb5>] i915_gem_do_execbuffer+0xba5/0x1260
[ 6379.741006] PGD babab067 PUD bb435067 PMD 0
[ 6379.741010] Oops: 0002 [#1] PREEMPT SMP
[ 6379.741014] last sysfs file: /sys/devices/pci0000:00/0000:00:1c.2/0000:06:00.0/ieee80211/phy0/rfkill0/state
[ 6379.741017] CPU 1
[ 6379.741021] Pid: 2186, comm: X Not tainted 2.6.33-rc4-00399-g24bc7347 #142 M11D/ESPRIMO Mobile M9400
[ 6379.741023] RIP: 0010:[<ffffffff8122ddb5>] [<ffffffff8122ddb5>] i915_gem_do_execbuffer+0xba5/0x1260
[ 6379.741027] RSP: 0018:ffff8800b9047b78  EFLAGS: 00213206
[ 6379.741029] RAX: 0000000000000000 RBX: 000000000000004f RCX: ffff880098cac800
[ 6379.741032] RDX: ffff880098caca78 RSI: ffff8800b9047c98 RDI: ffff880098cd6540
[ 6379.741034] RBP: ffff8800b9047c78 R08: ffffffff814b96b5 R09: 0000000000000006
[ 6379.741036] R10: 0000000000000000 R11: 0000000000000003 R12: 000000000000004e
[ 6379.741038] R13: 00000000fffffff7 R14: 0000000000000000 R15: 0000000000000001
[ 6379.741041] FS:  0000000000000000(0000) GS:ffff880001900000(0063) knlGS:00000000f72636c0
[ 6379.741043] CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
[ 6379.741041] FS:  0000000000000000(0000) GS:ffff880001900000(0063) knlGS:00000000f72636c0
[ 6379.741043] CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
[ 6379.741045] CR2: 00000000000000a0 CR3: 00000000b9000000 CR4: 00000000000006e0
[ 6379.741048] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 6379.741050] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 6379.741052] Process X (pid: 2186, threadinfo ffff8800b9046000, task ffff8800bb5d8000)
[ 6379.741054] Stack:
[ 6379.741055]  ffffc90023f57000 ffffc90023f56fff ffffc90023f56fff ffffc90023f55000
[ 6379.741059] <0> ffff8800b9047c98 ffff8800bb43c840 ffff8800bf1de800 ffff8800bf1de820
[ 6379.741063] <0> ffff8800b9047bd8 ffff880098cac800 0000000000000000 0000000000000002
[ 6379.741068] Call Trace:
[ 6379.741072]  [<ffffffff8122e6cb>] ?  i915_gem_execbuffer+0x6b/0x370
[ 6379.741077]  [<ffffffff810a5f52>] ? __vmalloc_node+0xa2/0xb0
[ 6379.741080]  [<ffffffff8122e6cb>] ?  i915_gem_execbuffer+0x6b/0x370
[ 6379.741083]  [<ffffffff8122e816>] i915_gem_execbuffer+0x1b6/0x370
[ 6379.741086]  [<ffffffff8120cd55>] drm_ioctl+0x1d5/0x460
[ 6379.741089]  [<ffffffff8122e660>] ?  i915_gem_execbuffer+0x0/0x370
[ 6379.741093]  [<ffffffff81248c35>] i915_compat_ioctl+0x45/0x50
[ 6379.741097]  [<ffffffff810f1659>] compat_sys_ioctl+0xa9/0x1570
[ 6379.741102]  [<ffffffff810b1d5c>] ? vfs_read+0x13c/0x1a0
[ 6379.741106]  [<ffffffff81028424>] sysenter_dispatch+0x7/0x2b
[ 6379.741108] Code: 08 85 c0 74 52 31 db 0f 1f 80 00 00 00 00 48 63 c3 48 8b
8d 68 ff ff ff 48 8d 14 c1 48 8b 02 48 85 c0 74 25 48 8b 80 80 00 00 00 <c7> 80
a0 00 00 00 00 00 00 00 48 8b 3a 48 85 ff 74 0c 48 c7 c6
[ 6379.741142] RIP  [<ffffffff8122ddb5>] i915_gem_do_execbuffer+0xba5/0x1260
[ 6379.741145]  RSP <ffff8800b9047b78>
[ 6379.741147] CR2: 00000000000000a0
[ 6379.741159] ---[ end trace 0598809afa4c31db ]---
Reported-by: NMatej Laitl <strohel@gmail.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

Suggested-by: NChris Wilson <chris@chris-wilson.co.uk&gt;>
Acked-by: Jesse Barnes <jbarnes@virtuousgeek.org> (in principle)
Signed-off-by: NEric Anholt <eric@anholt.net>

When we setup buffer for display plane, we'll check any pending
required GPU flush and possible make interruptible wait for flush
complete. But that wait would be most possibly to fail in case of
signals received for X process, which will then fail modeset process
and put display engine in unconsistent state. The result could be
blank screen or CPU hang, and DDX driver would always turn on outputs
DPMS after whatever modeset fails or not.

So this one creates new helper for setup display plane buffer, and
when needing flush using uninterruptible wait for that.

This one should fix bug like https://bugs.freedesktop.org/show_bug.cgi?id=24009.
Also fixing mode switch stress test on Ironlake.
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NEric Anholt <eric@anholt.net>

drm_pci_alloc() has input of address mask for setting pci dma
mask on the device, which should be properly setup by drm driver.
And leave it as a param for drm_pci_alloc() would cause confusion
or mistake would corrupt the correct dma mask setting, as seen on
intel hw which set wrong dma mask for hw status page. So remove
it from drm_pci_alloc() function.
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

As pinning (allocating and binding GTT memory) does not actually invoke
GPU commands, it is safe, and indeed is attempted, during resumption
from suspension:

  [drm:intel_init_clock_gating] *ERROR* failed to pin power context: -16
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reported-by: NHugh Dickins <hugh.dickins@tiscali.co.uk>
Cc: stable@kernel.org
Signed-off-by: NEric Anholt <eric@anholt.net>

This patch adds a new execbuf ioctl, execbuf2, for use by clients that
want to control fence register allocation more finely.  The buffer
passed in to the new ioctl includes a new relocation type to indicate
whether a given object needs a fence register assigned for the command
buffer in question.

Compatibility with the existing execbuf ioctl is implemented in terms
of the new code, preserving the assumption that fence registers are
required for pre-965 rendering commands.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
[ickle: Remove pre-emptive clear_fence_reg()]
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NKristian Høgsberg <krh@bitplanet.net>
[anholt: Removed dmesg spam]
Signed-off-by: NEric Anholt <eric@anholt.net>

i915_gem_object_unbind had the ordering wrong. The other user,
i915_gem_object_put_fence_reg already has the correct ordering.

Results was usually corrupted pixmaps, especially garbled font glyphs
after a suspend/resume (because this evicts everything).

I'm still waiting for the feedback from the bug-reporters, but
because this obviously fixes a bug (at least for me) I'm already
submitting it.

Bugzilla: http://bugs.freedesktop.org/show_bug.cgi?id=25406Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NEric Anholt <eric@anholt.net>
CC: stable@kernel.org

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: stable@kernel.org
Signed-off-by: NEric Anholt <eric@anholt.net>

IGD* isn't a useful name.  Replace with the codenames, as sourced from
pci.ids.
Signed-off-by: NAdam Jackson <ajax@redhat.com>
[anholt: Fixed up for merge with pineview/ironlake changes]
Signed-off-by: NEric Anholt <eric@anholt.net>

These are handled by the error return being propagated to user-space and
do not any add any information to the original error, so are useless.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

That is "success", "unknown", "through", "performance", "[re|un]mapping"
, "access", "default", "reasonable", "[con]currently", "temperature"
, "channel", "[un]used", "application", "example","hierarchy", "therefore"
, "[over|under]flow", "contiguous", "threshold", "enough" and others.
Signed-off-by: NAndré Goddard Rosa <andre.goddard@gmail.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

Acked-by: NJakob Bornecrantz <jakob@vmware.com>
Acked-by: NThomas Hellström <thomas@shipmail.org>
Review-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NJesse "Orange Smoothie" Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NKristian Høgsberg <krh@bitplanet.net>
Signed-off-by: NEric Anholt <eric@anholt.net>

Execbufs involve quite a bit of payload, to the extent that cache misses
show up in the profiles here, and a suspicion that some of those cachelines
may get evicted and then reloaded in the subsequent copy.

This is still abstracted like drm_calloc_large since we want to check for
size overflow, and because we want to choose between kmalloc and vmalloc
on the fly.  cairo's interface for malloc-with-calloc's-args was used as
the model.
Signed-off-by: NEric Anholt <eric@anholt.net>

Replace the DRM_DEBUG with DRM_DEBUG_DRIVER in generic i915 driver.
Then the debug info can be obtained by adding the boot option of
"drm.debug=0x02".

At the same time the debug info in increase/decrease clock is also
printed by using DRM_DEBUG_DRIVER instead of DRM_DEBUG_KMS.
Signed-off-by: NZhao Yakui <yakui.zhao@intel.com>
Signed-off-by: NEric Anholt <eric@anholt.net>

It's not needed anymore.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NEric Anholt <eric@anholt.net>

As long as the gpu can keep up, neither the cpu (waiting for gpu)
nore the gpu (waiting for vblank to do an overlay flip) stalls.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NEric Anholt <eric@anholt.net>

This just waits until the hw passed the current ring position with
cmd execution. This slightly changes the existing i915_wait_request
function to make uninterruptible waiting possible - no point in
returning to userspace while mucking around with the overlay, that
piece of hw is just too fragile.

Also replace a magic 0 with the symbolic constant (and kill the then
superflous comment) while I was looking at the code.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NEric Anholt <eric@anholt.net>

If we trigger a tracepoint for batch buffer submission, it is a reasonable
assumption that we wish to also trace the batch buffer completion. So in
order to capture the completion events, we need to enable irqs... However,
we cannot rely on the completion event to disable the irq later, so we
defer the irq disable to the retire request.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

We increment the seqno number between submitting the batch buffer and
the flush/interrupt that demarcates its end, so the tracepoint needs to
reference the incremented value to match the completion event.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

During a page fault and rebinding the buffer there exists a window for a
signal to arrive during the i915_wait_request() and trigger a
ERESTARTSYS. This used to be handled by returning SIGBUS and thereby
killing the application. Try 'cairo-perf-trace & cairo-test-suite' and
watch X go boom!

The solution as suggested by H. Peter Anvin is to simply return NOPAGE and
leave the higher layers to spot we did not fill the page and resubmit
the page fault.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: stable@kernel.org
[anholt: Mostly squash it with another commit]

Only allow the user to mmap buffers that have not been marked as
purgeable.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

In order to correctly prevent the invalid reuse of a purged buffer, we
need to track such events and warn the user before something bad
happens.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>