Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

On i830 if the tail pointer is set to within 2 cachelines of the end of
the buffer, the chip may hang. So instead if the tail were to land in
that location, we pad the end of the buffer with NOPs, and start again
at the beginning.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

In order to enforce the correct memory barriers for irq get/put, we need
to perform the actual counting using atomic operations.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

If the tail advances beyond the autoreport HEAD value, then we need to
fallback to an uncached read of the HEAD register in order to ascertain
the correct amount of remaining space in the ringbuffer.
Reported-by: NFang, Xun <xunx.fang@intel.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=32259Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

The bulk of the change is to convert the growing list of rings into an
array so that the relationship between the rings and the semaphore sync
registers can be easily computed.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

This makes the various rings more consistent by removing the anomalous
handing of the rendering ring execbuffer dispatch.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

A glorified s/obj_priv/obj/ with a net reduction of over a 100 lines and
many characters!
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Before reading ring register, set FORCE_WAKE bit to prevent GT core
power down to low power state, otherwise we may read stale values.
Signed-off-by: NZou Nan hai <nanhai.zou@intel.com>
[ickle: added a udelay which seemed to do the trick on my SNB]
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

On some stepping of SNB cpu, the first command to be parsed in BLT
command streamer should be MI_BATCHBUFFER_START otherwise the GPU
may hang.

(cherry picked from commit 8d19215b)

Conflicts:

	drivers/gpu/drm/i915/intel_ringbuffer.c
	drivers/gpu/drm/i915/intel_ringbuffer.h
Signed-off-by: NZou Nan hai <nanhai.zou@intel.com>
Cc: stable@kernel.org
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

On some stepping of SNB cpu, the first command to be parsed in BLT
command streamer should be MI_BATCHBUFFER_START otherwise the GPU
may hang.
Signed-off-by: NZou Nan hai <nanhai.zou@intel.com>
[ickle: rebased for -next]
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

The render ring is not alone any more! And the other rings are just as
troublesome...
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Preparing the ringbuffer for adding new commands can fail (a timeout
whilst waiting for the GPU to catch up and free some space). So check
for any potential error before overwriting HEAD with new commands, and
propagate that error back to the user where possible.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

The ringbuffer keeps a pointer to the parent device, so we can use that
instead of passing around the pointer on the stack.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

... to prevent flush processing of an idle (or even absent) ring.

This fixes a regression during suspend from 87acb0a5.
Reported-and-tested-by: NAlexey Fisher <bug-track@fisher-privat.net>
Tested-by: NPeter Clifton <pcjc2@cam.ac.uk>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

This should fix the error along the reset path were we tried to clear the
tail register by setting it to 0, but were in fact setting it to the
current value and complaining when it did not reset to 0.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Based on an original patch by Zhenyu Wang, this initializes the BLT ring for
SandyBridge and enables support for user execbuffers.

Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

... before someone tries to use it. The code both calls
intel_ring_begin/advance() and open-codes the bookkeeping performed by
those two functions.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

With multiple rings generating requests independently, the outstanding
requests must also be track independently.
Reported-by: NWang Jinjin <jinjin.wang@intel.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=30380Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

It's the same code, essentially, so kill all copies safe one unified
version.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

All functions are extremely similar, so fold them into one generic
implementation.

This function isn't used anyway, because there's not yet a bsd ring
error state dumper.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Avoid cause latencies in other clients by not taking the global struct
mutex and moving the per-client request manipulation a local per-client
mutex. For example, this allows a compositor to schedule a page-flip
(through X) whilst an OpenGL application is monopolising the GPU.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

As we presume space is signed when computing and looking for wrap along,
make it so.
Reported-by: NOwain G. Ainsworth <zerooa@googlemail.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

We are not currently using it as intended, so remove the complication.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Always PAGE_SIZE and only complicates the code.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Documentation explicitly mentions that the ring registers are
designed to have the same offsets relative to a base registers.

Use this to fight the code beaurocratic in intel_ringbuffer.c.

No code changes in this patch, just the new definitions.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

As noted by Zhenyu, we can now simply replace the existing advance hook
by calling the new set_tail function pointer directly.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

This is prepared for video codec ring buffer on Sandybridge. It is
needed to read/write more than one register to move the tail pointer of
the video codec ring on Sandybridge.
Signed-off-by: NXiang, Haihao <haihao.xiang@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Introduce intel_init_render_ring_buffer(), intel_init_bsd_ring_buffer
for ring initialization.
Signed-off-by: NXiang, Haihao <haihao.xiang@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

When flushing the GPU domains,we emit a flush on *both* rings, even
though they share a unified cache. Only emit the flush on the currently
active ring.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

This can always be re-added should somebody find a use...
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Previously I thought that one interrupt per batchbuffer should be
enough. Now tedious benchmarking showed this to be wrong.

Therefore track whether any commands have been isssued with a future
seqno (like pipelined fencing changes or flushes). If this is the case
emit a request before issueing the batchbuffer.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

This will be used by the eviction logic to maintain fairness between the
rings.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>