With the new "standardized" sysfs interfaces we need to be a bit more
careful about setting the RPS values.

Because the sysfs code and the rps workqueue can run at the same time,
if the sysfs setter wins the race to the mutex, the workqueue can come
in and set a value which is out of range (ie. we're no longer protecting
by RPINTLIM).

I was not able to actually make this error occur in testing.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Rather than have multiple data structures for describing our page layout
in conjunction with the array of pages, we can migrate all users over to
a scatterlist.

One major advantage, other than unifying the page tracking structures,
this offers is that we replace the vmalloc'ed array (which can be up to
a megabyte in size) with a chain of individual pages which helps reduce
memory pressure.

The disadvantage is that we then do not have a simple array to iterate,
or to access randomly. The common case for this is in the relocation
processing, which will typically fit within a single scatterlist page
and so be almost the same cost as the simple array. For iterating over
the array, the extra function call could be optimised away, but in
reality is an insignificant cost of either binding the pages, or
performing the pwrite/pread.

v2: Fix drm_clflush_sg() to not invoke wbinvd as well! And fix the
trivial compile error from rebasing.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently we've only frobbed this bit at irq_init time, but did
not restore it at resume time. Move it to the gen3 clock gating
function to fix this.

Notice while reading through code.

Cc: stable@vger.kernel.org (for 3.5 only)
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Like with the equivalent change for gen6+ rps state, this helps in
clarifying the code (and in fixing a few places that have fallen through
the cracks in the locking review).
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Otherwise it just won't compile ...
Reported-by: NFengguang Wu <fengguang.wu@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@gmail.com>

Using the extracted INSTDONE reading, and our new register definitions,
update our hangcheck detection and error collection to use it. This
primarily means changing == to memcmp, and changing = to memcpy.
Hopefully this will give more info on error dump, and provide more
accurate hangcheck detection (both are actually TBD).

Also, remove the reading of instdone1 from the ring error collection
function, and just crap everything in capture_error_state (that could be
split into a separate patch if it wasn't so trivial).

v2: Now assuming i915_get_extra_instdone does the memset we can clean up the
code a bit (Jani)

v3: use ARRAY_SIZE as requested earlier by Jani (didn't change sizeof)
Updated commit msg

Cc: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

INSTDONE is used in many places, and it varies from generation to
generation. This provides a good reason for us to extract the logic to
read the relevant information.

The patch has no functional change. It's prep for some new stuff.

v2: move the memset inside of i915_get_extra_instdone (Jani)
v3,4: bugs caught by (Jani)
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

ERR_INT can generate interrupts. However since most of the conditions seem
quite fatal the patch opts to simply report it in error state instead of
adding more complexity to the interrupt handler for little gain (the
bits are sticky anyway).
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Tested-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NAntti Koskipaa <antti.koskipaa@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

When dealing with a working set larger than the GATT, or even the
mappable aperture when touching through the GTT, we end up with evicting
objects only to rebind them at a new offset again later. Moving an
object into and out of the GTT requires clflushing the pages, thus
causing a double-clflush penalty for rebinding.

To avoid having to clflush on rebinding, we can track the pages as they
are evicted from the GTT and only relinquish those pages on memory
pressure.

As usual, if it were not for the handling of out-of-memory condition and
having to manually shrink our own bo caches, it would be a net reduction
of code. Alas.

Note: The patch also contains a few changes to the last-hope
evict_everything logic in i916_gem_execbuffer.c - we no longer try to
only evict the purgeable stuff in a first try (since that's superflous
and only helps in OOM corner-cases, not fragmented-gtt trashing
situations).

Also, the extraction of the get_pages retry loop from bind_to_gtt (and
other callsites) to get_pages should imo have been a separate patch.

v2: Ditch the newly added put_pages (for unbound objects only) in
i915_gem_reset. A quick irc discussion hasn't revealed any important
reason for this, so if we need this, I'd like to have a git blame'able
explanation for it.

v3: Undo the s/drm_malloc_ab/kmalloc/ in get_pages that Chris noticed.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: Split out code movements and rant a bit in the commit message
with a few Notes. Done v2]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Avoid the forcewake overhead when simply retiring requests, as often the
last seen seqno is good enough to satisfy the retirment process and will
be promptly re-run in any case. Only ensure that we force the coherent
seqno read when we are explicitly waiting upon a completion event to be
sure that none go missing, and also for when we are reporting seqno
values in case of error or debugging.

This greatly reduces the load for userspace using the busy-ioctl to
track active buffers, for instance halving the CPU used by X in pushing
the pixels from a software render (flash). The effect will be even more
magnified with userptr and so providing a zero-copy upload path in that
instance, or in similar instances where X is simply compositing DRI
buffers.

v2: Reverse the polarity of the tachyon stream. Daniel suggested that
'force' was too generic for the parameter name and that 'lazy_coherency'
better encapsulated the semantics of it being an optimization and its
purpose. Also notice that gen6_get_seqno() is only used by gen6/7
chipsets and so the test for IS_GEN6 || IS_GEN7 is redundant in that
function.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We change the drps/ips sw/hw state from different callers: Our own irq
handler, the external intel-ips module and from process context. Most
of these callers don't take any lock at all.

Protect everything by making the mchdev_lock irqsave and grabbing it in
all relevant callsites. Note that we have to convert a few sleeps in the
drps enable/disable code to delays, but alas, I'm not volunteering to
restructure the code around a few work items.

For paranoia add a spin_locked assert to ironlake_set_drps, too.

v2: Move one access inside the lock protection. Caught by the
dev_priv->ips mass-rename ...

v3: Resolve rebase conflict.
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Like all the other drps/ips stuff. Hence add the corresponding check,
give the function a preciser prefix and move the single reg clearing into
the rps handling function, too.
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This way it's easier so see what belongs together, and what is used
by the ilk ips code. Also add some comments that explain the locking.

Note that (cur|min|max)_delay need to be duplicated, because
they're also used by the ips code.

v2: Missed one place that the dev_priv->ips change caught ...
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Originally I had a macro specifically for DPF support, and Daniel, with
good reason asked me to change it to this. It's not the way I would have
gone (and indeed I didn't), but for now there is no distinction as all
platforms with L3 also have DPF.

Note: The good reasons are that dpf is a l3$ feature (at least on
currrent hw), hence I don't expect one to go without the other.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
[danvet: added note]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

As we always flush the GPU cache prior to emitting the breadcrumb, we no
longer have to worry about the deferred flush causing the
pending_gpu_write to be delayed. So we can instead utilize the known
last_write_seqno to hopefully minimise the wait times.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Having had to dive into the bspec to understand what each stage of the
workaround meant, and how that the ring broadcasting IDLE corresponded
with the GT powering down the ring (i.e. rc6) add comments to aide
the next reader.

And since the register "is used to control all aspects of PSMI and power
saving functions" that makes it quite interesting to inspect with
regards to RC6 hangs, so add it to the error-state.

v2: Rediscover the piece of magic, set the RNCID to 0 before waiting for
the ring to wake up.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

After banging my head against this for the past few months, I still
don't see how this could possible race under the premise that once an
irq bit is masked in PM_IMR and reset in PM_IIR it won't show up again
until we unmask it in PM_IMR.

Still, we have reports of this being seen in the wild. Now Bspec has
this little bit of lovely language in the PMIIR register:

Public SNB Docs, Vol3Part2, 2.5.14 "PMIIR":

"For each bit, the IIR can store a second pending interrupt if two or
more of the same interrupt conditions occur before the first condition
is cleared. Upon clearing the interrupt, the IIR bit will momentarily
go low, then return high to indicate there is another interrupt
pending."

Now if we presume that PMIMR only prevent new interrupts from being
queued, we could easily end up masking an interrupt and clearing it,
but the 2nd pending interrupt setting the bit in PMIIR right away
again. Which leads, the next time the irq handler runs, to hitting the
WARN.

Also, no bad side effects of this have ever been reported. And we've
tracked down our issues with the gpu turbo getting stuck to bogus
interrupt generation limits in th RPLIMIT register.

So let's just rip out this WARN as bogus and call it a day. The only
shallow thing here is that this 2-deep irq queue in the hw makes you
wonder how racy the windows irq handler is ...

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=42907
Cc: stable@vger.kernel.org
Acked-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

And restructure the IRQ handling a little.  We can use pipestat for most
things, and make sure we don't affect pipe events when enabling and
disabling vblank interupts.

We can leave vblank interrupts masked but enabled so we're not dependent
on the first client to toggle the disable timer.  We can also mask all
render based interrupts, since the ring code will handle unmasking them
for us.

v2: roll in vblank masking, remove unneeded variable (Daniel)
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NBen Widawsky <ben@bwidawsk.net>

Cougar/Panther Point redefine the bits in SDEIIR pretty completely.
This function is just debugging, but if we're debugging we probably want
to be told accurate things instead of lies.

I'm told Lynx Point changes this yet more, but I have no idea how...

Note from Eugeni's review:

"For the record and for future enabling efforts, for LPT, bits 28-31
and 1-14 are gone since CPT/PPT (e.g., those must be zero). And there
is the bit 15 as a new addition, but we are not using it yet and
probably won't be using in foreseeable future."
Signed-off-by: NAdam Jackson <ajax@redhat.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=35103Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This fixes an (albeit really hard to hit) race resulting in an oops:
- The parity work get scheduled.
- We re-init the irq state and call INIT_WORK again.
- The workqueue code tries to run the work item and stumbles over a
  work item that should be on it's runlist.

Also initiliaze the work item unconditionally like all the others,
it's simpler.
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Notice by Fengguang Wu's automatic sparse checker.
Reported-by: NFengguang Wu <wfg@linux.intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The previous patch put all the code, and handlers in place. It should
now be safe to enable the parity error interrupt. The parity error must
be unmasked in both the GTIMR, and the CS IMR. Unfortunately, the docs
aren't clear about this; nevertheless it's the truth.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

On IVB hardware we are given an interrupt whenever a L3 parity error
occurs in the L3 cache. The L3 cache is used by internal GPU clients
only.  This is a very rare occurrence (in fact to test this I need to
use specially instrumented silicon).

When a row in the L3 cache detects a parity error the HW generates an
interrupt. The interrupt is masked in GTIMR until we get a chance to
read some registers and alert userspace via a uevent. With this
information userspace can use a sysfs interface (follow-up patch) to
remap those rows.

Way above my level of understanding, but if a given row fails, it is
statistically more likely to fail again than a row which has not failed.
Therefore it is desirable for an operating system to maintain a lifelong
list of failing rows and always remap any bad rows on driver load.
Hardware limits the number of rows that are remappable per bank/subbank,
and should more than that many rows detect parity errors, software
should maintain a list of the most frequent errors, and remap those
rows.

V2: Drop WARN_ON(IS_GEN6) (Jesse)
DRM_DEBUG row/bank/subbank on errror (Jesse)
Comment updates (Jesse)
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In order to avoid missed down-interrupts when coming out of RC6, it is
advised that we always reset the down-threshold upon a PM event. This is
due to that the PM unit goes through a little dance when coming out of
RC6, it first brings the GPU up at the lowest frequency then a short
time later it restores the thresholds. During that interval, the
down-interval may expire and the interrupt be suppressed.

Now aware of the dance taking place within the GPU when coming out of
RC6, one wonders what other writes need to be queued in the fifo buffer
in order to be properly sequenced; setting the RP state appears to be
one.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=44006Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Acked-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Note that gen3 is the only platform where we've got the bit
definitions right, hence the workaround of disabling sdvo hotplug
support on i945g/gm is not due to misdiagnosis of broken hotplug irq
handling ...
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: add some blurb about sdvo hotplug fail on i945g/gm I've
wondered about while reviewing.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Also as we set the HOTPLUG_EN to 0 during pre-install, we can simply set
it during post-install, and nor do we wish to enable unwanted hotplug
events.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In many places we wish to iterate over the rings associated with the
GPU, so refactor them to use a common macro.

Along the way, there are a few code removals that should be side-effect
free and some rearrangement which should only have a cosmetic impact,
such as error-state.

Note that this slightly changes the semantics in the hangcheck code:
We now always cycle through all enabled rings instead of
short-circuiting the logic.

v2: Pull in a couple of suggestions from Ben and Daniel for
intel_ring_initialized() and not removing the warning (just moving them
to a new home, closer to the error).
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
[danvet: Added note to commit message about the small behaviour
change, suggested by Ben Widawsky.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Haswell interrupts are mostly similar with Ivy Bridge, so we share same
routines with it.

This patch also simplifies the vblank counter handling for all the Gen5+
architectures.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We can take advantage that the PCH_IIR is a subordinate register to
reduce one of the required IIR reads, and that we only need to clear
interrupts handled to reduce the writes. And by simply tidying the code
we can reduce the line count and hopefully make it more readable.

v2: Split out the bugfix from the refactoring.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently the code re-reads PCH_IIR during the hotplug interrupt
processing. Not only is this a wasted read, but introduces a potential
for handling a spurious interrupt as we then may not clear all the
interrupts processed (since the re-read IIR may contains more interrupts
asserted than we clear using the result of the original read).
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Cc: stable@kernel.org
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Only half of them even cared, and it's always the same one.
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

- reduce the irq disabled section, even for a debugfs file this was
  way too long.
- always disable irqs when taking the lock.

v2: Thou shalt not mistake locking for reference counting, so:
- reference count the error_state to protect from concurent freeeing.
  This will be only really used in the next patch.
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

... and hide it in i915_dma.c.

This way all the legacy stuff dealing with READ_BREADCRUMB and
LP_RING and friends is in i915_dma.c.

v2: Rebase on top of Chris Wilson's rework irq handling code.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Let's just get this out of the way.

v2: Rebase against ENODEV changes.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We never supported dri1 on gen5+.

VLV never had that code, so no need to remove it.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

i915_dma.c contains most of the old dri1 horror-show, so move
the remaining bits there, too. The code has been removed and
the only thing left are some stubs to ensure that userspace
doesn't try to use this stuff. vblank_pipe_set only returns 0
without any side-effects, so we can even stub it out with
the canonical drm_noop.

v2: Rebase against ENODEV changes.
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

vblank_pipe was intended to be used for tracking DRI1 state. However,
the vblank_pipe reported to DRI1 is fixed to umask both pipes, and the
dev_priv->vblank_pipe unused and superfluous.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>