We have to do this manually. Somebody had a Great Idea.

I've measured speed-ups just a few percent above the noise level
(below 5% for the best case), but no slowdows. Chris Wilson measured
quite a bit more (10-20% above the usual snb variance) on a more
recent and better tuned version of sna, but also recorded a few
slow-downs on benchmarks know for uglier amounts of snb-induced
variance.

v2: Incorporate Ben Widawsky's preliminary review comments and
elaborate a bit about the performance impact in the changelog.

v3: Add a comment as to why we don't need to check the 3rd memory
channel.

v4: Fixup whitespace.
Acked-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NEric Anholt <eric@anholt.net>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It looks like the desktop variants of i915 and i945 also have the DCC
register to control dram channel interleave and cpu side bit6
swizzling.

Unfortunately internal Cspec/ConfigDB documentation for these ancient chips
have already been dropped and there seem to be no archives. Also
somebody thought the swizzling behaviour is surely a worthy secret to
keep and redacted any mention of these fields from the published Intel
datasheets.

I suspect the hw engineers were really proud of the page coloring
they've achieved in their first dual channel dram controller with
bit17 - after all Bspec explains in great length the optimal layout of
page frame numbers modulo 4 for the color and depth buffers, too.
Later on when they've started to work on VT-d they shamefully
discoverd their stupidity and tried to cover the tracks ...

Tested-by: Daniel Vetter <daniel.vetter@ffwll.ch> (i915g)
Tested-by: Pavel Ondračka <pavel.ondracka@email.cz> (i945g)
Tested-by: NChris Wilson <chris@chris-wilson.co.uk>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=42625Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Use the helper function already employed by the pwrite/pread
functions.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Fixes tests/gem_tiled_pread on my snb. I know, mesa doesn't use this
on gen6+, but I also hate failing testcases.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Align unfenced buffers on older hardware to the power-of-two object
size.  The docs suggest that it should be possible to align only to a
power-of-two tile height, but using the already computed fence size is
easier and always correct. We also have to make sure that we unbind
misaligned buffers upon tiling changes.

In order to prevent a repetition of this bug, we change the interface
to the alignment computation routines to force the caller to provide
the requested alignment and size of the GTT binding rather than assume
the current values on the object.
Reported-and-tested-by: NSitosfe Wheeler <sitsofe@yahoo.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=36326Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: stable@kernel.org
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Treat it like Ironlake and Sandy Bridge.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NKeith Packard <keithp@keithp.com>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Early gen3 and gen2 chipset do not have the relaxed per-surface tiling
constraints of the later chipsets, so we need to check that the GTT
alignment is correct for the new tiling. If it is not, we need to
rebind.
Reported-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

This reverts commit c2e0eb16.

As it turns out, userspace already depends upon being able to enable
tiling on existing bo which it promises to be large enough for its
purposes i.e. it will not access beyond the end of the last full-tile
row.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=35016Reported-and-tested-by: NKamal Mostafa <kamal@canonical.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

It looks like gen2 has a peculiar interleaved 2-row inter-tile
layout. Probably inherited from i81x which had 2kb tiles (which
naturally fit an even-number-of-tile-rows scheme to fit onto 4kb
pages). There is no other mention of this in any docs (also not
in the Intel internal documention according to Chris Wilson).

Problem manifests itself in corruptions in the second half of the
last tile row (if the bo has an odd number of tiles). Which can
only happen with relaxed tiling (introduced in a00b10c3).

So reject set_tiling calls that don't satisfy this constrain to
prevent broken userspace from causing havoc. While at it, also
check the size for newer chipsets.

LKML: https://lkml.org/lkml/2011/2/19/5Reported-by: NIndan Zupancic <indan@nul.nu>
Tested-by: NIndan Zupancic <indan@nul.nu>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Dave Airlie spotted that we had a potential bug should we ever rearrange
the drm_i915_gem_object so not the base drm_gem_object was not its first
member. He noticed that we often convert the return of
drm_gem_object_lookup() immediately into drm_i915_gem_object and then
check the result for nullity. This is only valid when the base object is
the first member and so the superobject has the same address. Play safe
instead and use the compiler to convert back to the original return
address for sanity testing.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Instead of reporting EIO upfront in the entrance of an ioctl that may or
may not attempt to use the GPU, defer the actual detection of an invalid
ioctl to when we issue a GPU instruction. This allows us to continue to
use bo in video memory (via pread/pwrite and mmap) after the GPU has hung.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

With this change, every batchbuffer can use all available fences (save
pinned and scanout, of course) without ever stalling the gpu!

In theory. Currently the actual pipelined update of the register is
disabled due to some stability issues. However, just the deferred update
is a significant win.

Based on a series of patches by Daniel Vetter.

The premise is that before every access to a buffer through the GTT we
have to declare whether we need a register or not. If the access is by
the GPU, a pipelined update to the register is made via the ringbuffer,
and we track the last seqno of the batches that access it. If by the
CPU we wait for the last GPU access and update the register (either
to clear or to set it for the current buffer).

One advantage of being able to pipeline changes is that we can defer the
actual updating of the fence register until we first need to access the
object through the GTT, i.e. we can eliminate the stall on set_tiling.
This is important as the userspace bo cache does not track the tiling
status of active buffers which generate frequent stalls on gen3 when
enabling tiling for an already bound buffer.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

A glorified s/obj_priv/obj/ with a net reduction of over a 100 lines and
many characters!
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

As we may bind an object with the correct alignment, but with an invalid
size, it may pass the current checks on whether the object may be reused
with a fence.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

So long as we adhere to the fence registers rules for alignment and no
overlaps (including with unfenced accesses to linear memory) and account
for the tiled access in our size allocation, we do not have to allocate
the full fenced region for the object. This allows us to fight the bloat
tiling imposed on pre-i965 chipsets and frees up RAM for real use. [Inside
the GTT we still suffer the additional alignment constraints, so it doesn't
magic allow us to render larger scenes without stalls -- we need the
expanded GTT and fence pipelining to overcome those...]
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

So remove the redundant bit in the capabilities block and
s/IS_IRONLAKE/IS_GEN5/.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Owain Ainsworth reported an issue between the interaction of the
hangcheck and userspace immediately (and permanently) falling back to
s/w rasterisation. In order to break the mutex and begin resetting the
GPU, we must abort the current operation (usually within the wait) and
climb sufficiently far back up the call chain to drop the mutex. In his
implementation, Owain has a loop within the ioctl handler to detect the
hang and then sleep until the error handler has run. I've chosen to
return to userspace and report an EAGAIN which should trigger the
userspace ioctl handler to repeat the call (simply because it felt less
invasive...). Before hitting a wedged GPU, we then wait upon completion
of the error handler.
Reported-by: NOwain G. Ainsworth <zerooa@googlemail.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Avoid confusion between i965g meaning broadwater and the gen4+ chipset
families.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

As we currently may need to acquire a fence register during a modeset,
we need to be able to do so in an uninterruptible manner. So expose that
parameter to the callers of the fence management code.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Our usage of kmap() cannot return NULL here, so remove the unnecessary
error handling.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

This is consistent with trying to access a filename that not exist
within a directory which is a good analogy here. The main reason for the
change is that it is easy to confuse the error code of EBADF as an
performing an ioctl on an invalid file descriptor (rather than an
unknown object).
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDave Airlie <airlied@redhat.com>

We generally issue an error message at the point of failure, and so this
warning with a fairly pointless stacktrace is superfluous and ugly.
Needless to say, the common trigger for this WARN happens to be EIO
where this is pure noise.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

Makes no sense and complicates matters for pipelined tiling changes.
So don't allow it and return -EBUSY.

v2: Fix reference leak. Thanks to Owain Ainsworth for spotting this.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NEric Anholt <eric@anholt.net>

Current code is definitely crap: Largest pitch allowed spills into
the TILING_Y bit of the fence registers ... :(

I've rewritten the limits check under the assumption that 3rd gen hw
has a 3d pitch limit of 8kb (like 2nd gen). This is supported by an
otherwise totally misleading XXX comment.

This bug mostly resulted in tiling-corrupted pixmaps because the kernel
allowed too wide buffers to be tiled. Bug brought to the light by the
xf86-video-intel 2.11 release because that unconditionally enabled
tiling for pixmaps, relying on the kernel to check things. Tiling for
the framebuffer was not affected because the ddx does some additional
checks there ensure the buffer is within hw-limits.

v2: Instead of computing the value that would be written into the
hw fence registers and then checking the limits simply check whether
the stride is above the 8kb limit. To better document the hw, add
some WARN_ONs in i915_write_fence_reg like I've done for the i830
case (using the right limits).
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=27449Tested-by: NAlexander Lam <lambchop468@gmail.com>
Cc: stable@kernel.org
Signed-off-by: NEric Anholt <eric@anholt.net>

This is a purely cosmetic change to make changes in this area easier.
And hey, it's not only clearer and typechecked, but actually shorter,
too!

[anholt: To clarify, this is a change to let us later make
drm_i915_gem_object subclass drm_gem_object, instead of having
drm_gem_object have a pointer to i915's private data]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: NDave Airlie <airlied@gmail.com>
Signed-off-by: NEric Anholt <eric@anholt.net>

Bug introduced in

commit 10ae9bd2
Author: Daniel Vetter <daniel.vetter@ffwll.ch>
Date:   Mon Feb 1 13:59:17 2010 +0100

    drm/i915: blow away userspace mappings before fence change

The problem is that when there's no fence reg assigned and the object
is mapped at a fenceable offset in the gtt, the userspace mappings won't
be torn down. Which happens on untiled->tiled transition quite often
on 4th gen and later because there fencing does not have any special
alignment constraints (as opposed to 2nd and 3rd gen on which I've tested
the original commit).

Bugzilla: http://bugs.freedesktop.org/show_bug.cgi?id=26993Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Tested-by: Eric Anholt <eric@anholt.net> (fixes OpenArena)
Signed-off-by: NEric Anholt <eric@anholt.net>

Signed-off-by: NEric Anholt <eric@anholt.net>

i915_gem_object_fenceable was mostly just a repeat of the
i915_gem_object_fence_offset_ok, but also checking the size (which was
checkecd when we allowed that BO to be tiled in the first place). So
instead, export the latter function and use it in place.
Signed-Off-By: NOwain G. Ainsworth <oga@openbsd.org>
Signed-off-by: NEric Anholt <eric@anholt.net>

This aligns it with the other user of i915_gem_clear_fence_reg,
which blows away the mapping before changing the fence reg.

Only affects userspace if it races against itself when changing
tiling parameters, i.e. behaviour is undefined, anyway.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEric Anholt <eric@anholt.net>

As we need more and more controls within MCHBAR for memory config and
power management, this trys to keep MCHBAR enabled from driver load and
only tear down in driver unload.
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NEric Anholt <eric@anholt.net>

Mostly obvious simplifications.

The i915 pread/pwrite ioctls, intel_overlay_put_image and
nouveau_gem_new were incorrectly using the locked versions
without locking: this is also fixed in this patch.
Signed-off-by: NLuca Barbieri <luca@luca-barbieri.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This patch adds a new execbuf ioctl, execbuf2, for use by clients that
want to control fence register allocation more finely.  The buffer
passed in to the new ioctl includes a new relocation type to indicate
whether a given object needs a fence register assigned for the command
buffer in question.

Compatibility with the existing execbuf ioctl is implemented in terms
of the new code, preserving the assumption that fence registers are
required for pre-965 rendering commands.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
[ickle: Remove pre-emptive clear_fence_reg()]
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NKristian Høgsberg <krh@bitplanet.net>
[anholt: Removed dmesg spam]
Signed-off-by: NEric Anholt <eric@anholt.net>

IGD* isn't a useful name.  Replace with the codenames, as sourced from
pci.ids.
Signed-off-by: NAdam Jackson <ajax@redhat.com>
[anholt: Fixed up for merge with pineview/ironlake changes]
Signed-off-by: NEric Anholt <eric@anholt.net>

Replace the DRM_DEBUG with DRM_DEBUG_DRIVER in generic i915 driver.
Then the debug info can be obtained by adding the boot option of
"drm.debug=0x02".

At the same time the debug info in increase/decrease clock is also
printed by using DRM_DEBUG_DRIVER instead of DRM_DEBUG_KMS.
Signed-off-by: NZhao Yakui <yakui.zhao@intel.com>
Signed-off-by: NEric Anholt <eric@anholt.net>

The driver gets the bridge device in a number of places, upcoming
vga arb code paths need the bridge device, however they need it in
under a lock, and the pci lookup can allocate memory. So clean
this code up before then and get the bridge once for the driver lifetime.
Signed-off-by: NDave Airlie <airlied@redhat.com>

It seems that on IGDNG the same swizzling setup always applys.
And front buffer tiling needs to set address swizzle in display
arb control too.

Fix plane tricle feed setting in v1 which should be disable bit,
and always setup address swizzle to let hardware care for buffer
tiling in all cases.
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NEric Anholt <eric@anholt.net>

As of 52dc7d32, we could leave an old
linear GTT mapping in place, so that apps trying to GTT-mapped write in
tiled data wouldn't get the fence added, and garbage would get displayed.
Signed-off-by: NEric Anholt <eric@anholt.net>

Signed-off-by: NKeith Packard <keithp@keithp.com>

The fence register value also depends upon the stride of the object, so we
need to clear the fence if that is changed as well.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[anholt: Added 8xx and 965 paths, and renamed the confusing
i915_gem_object_tiling_ok function to i915_gem_object_fence_offset_ok]
Signed-off-by: NEric Anholt <eric@anholt.net>

Using the new PNP resource checking code, this patch allows the i915
driver to allocate MCHBAR space if needed and use the BAR to determine
current memory settings.

[apw@canonical.com: moved to the new generic PNP resource interface]
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NAndy Whitcroft <apw@canonical.com>
Signed-off-by: NEric Anholt <eric@anholt.net>

failure to update-index after git-am --reject to hand-apply
Signed-off-by: NEric Anholt <eric@anholt.net>