- Take the dev->struct_mutex around access the corresponding state
  (and adjusting the rps hw state).
- Add an assert to gen6_set_rps to ensure we don't forget about this
  in the future.
- Don't set up the min/max_freq files if it doesn't apply to the hw.
  And do the same for the gen6+ cache sharing file while at it.

v2: Move the gen6+ checks into the read/write callbacks. Thanks to the
awesome drm midlayer we can't check that when registering the debugfs
files, because the driver is not yet fully set up, specifically the
->load callback hasn't run yet.

Oh how I despise this disaster ...

v3: Also add a WARN_ON(mutex_is_locked) in set_rps to check the
locking.

v4: Use mutex_lock_interruptible, suggested by Chris Wilson.

Reviewed-by: Ben Widawsky <ben@bwidawsk.net> (for v2)
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The update_gfx_val function called from mark_busy wasn't taking the
mchdev_lock, as it should have. Also sprinkle a few spinlock asserts
over the code to document things better.

Things are still rather confusing, especially since a few variables
in dev_priv are used by both the gen6+ rps code and the ilk ips code.
But protected by totally different locks. Follow-on patches will clean
that up.

v2: Don't add a deadlock ... hence split up update_gfx_val into a
wrapper that grabs the lock and an internal __ variant for callsites
within intel_pm.c that already have taken the lock.

v3: Mark the internal helper as static, noticed by Ben Widawsky.

v4: Damien Lespiau had questions about the safety of the ips setup
sequence, explain in a comment why it works.
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In commit

commit 20b46e59
Author: Daniel Vetter <daniel.vetter@ffwll.ch>
Date:   Thu Jul 26 11:16:14 2012 +0200

    drm/i915: Only set the down rps limit when at the loweset frequency

The computation for the new desired frequency was extracted, but since
the desired frequency was passed-by value, the adjustments didn't
propgate back. Fix this.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We believe to have squashed all issues around the gen6+ rps interrupt
generation and why the gpu sometimes got stuck. With that cleared up,
there's no user left for the sanitize_pm infrastructure, so let's just
rip it out.

Note that 'intel_reg_write 0xa014 0x13070000' is the w/a if we find
ourselves stuck again.
Acked-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The power docs say that when the gt leaves rc6, it is in the lowest
frequency and only about 25 usec later will switch to the frequency
selected in GEN6_RPNSWREQ. If the downclock limit expires in that
window and the down limit is set to the lowest possible frequency, the
hw will not send the down interrupt. Which leads to a too high gpu
clock and wasted power.

Chris Wilson already worked on this with

commit 7b9e0ae6
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date:   Sat Apr 28 08:56:39 2012 +0100

    drm/i915: Always update RPS interrupts thresholds along with
    frequency

but got the logic inverted: The current code set the down limit as
long as we haven't reached it. Instead of only once with reached the
lowest frequency.

Note that we can't always set the downclock limit to 0, because
otherwise the hw will keep on bugging us with downclock request irqs
once the lowest level is reached.

For similar reasons also always set the upclock limit, otherwise the
hw might poke us again with interrupts.

v2: Chris Wilson noticed that the limit reg is also computed in
sanitize_pm. To avoid duplication, extract the code into a common
function.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

As suggested by Daniel, rip out the independent timers for device and
crtc busyness and integrate the manual powermanagement of the display
engine into the GEM core and its request tracking. The benefits are that
the code is a lot smaller, fewer moving parts and should fit more neatly
into the overall activity tracking of the driver.

v2: Complete overhaul and removal of the racy timers and workers.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This initializes power wells within the modeset_init_hw routine.
Testing has shown that this works for both driver load time and for
suspend-resume code paths.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

There is little point waking up every 10ms to service an interrupt which
we then promptly ignore. So only program the the PMIER to enable
interrupts for those events which we do handle, not all of them!
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Cc: Eugeni Dodonov <eugeni.dodonov@intel.com>
Cc: Ben Widawsky <ben@bwidawsk.net>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This pollutes dmesg output even if we do not have FBC for the device, so
move the DRM_DEBUG_KMS statement lower.

v2: just kill the message as suggested by Daniel.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

While creating the new enable/disable_gt_powersave functions in

commit 8090c6b9
Author: Daniel Vetter <daniel.vetter@ffwll.ch>
Date:   Sun Jun 24 16:42:32 2012 +0200

    drm/i915: wrap up gt powersave enabling functions

I've botched up the handling of ironlake_disable_rc6. Fix this up by
calling it at the right place. Note though that ironlake_disable_rc6
does a bit more than just disabling rc6 - it also tears down all the
allocated context objects.

Hence we need to move intel_teardown_rc6 out and directly call it from
intel_modeset_cleanup.

Also properly mark ironlake_enable_rc6 as static and kill the un-used
declaration in i915_drv.h.

Note: In review a question popped out why disable_rc6 also tears down
the backing object and why we should move that out - it's simply for
consistency with gen6+ rps code, which does it that way.

Cc: Ben Widawsky <ben@bwidawsk.net>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This commit moves force wake support routines into intel_pm modules, and
exports the gen6_gt_check_fifodbg routine (used in I915_READ).
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Acked-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

For Haswell, on some of the early hardware revisions, it is possible to
run into issues when RC6 state is enabled and when pipes change state.

v2: add comment saying that this is for early revisions only.

v3: beautify as suggested by Daniel Vetter.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Acked-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This is based on Ivy Bridge clock gating for now, but is subject to
changes in the future.

Note: Compared to the ivb clock gating this drops the the IDICOS
medium uncore sharing tuned in

commit 20848223
Author: Ben Widawsky <ben@bwidawsk.net>
Date:   Fri May 4 18:58:59 2012 -0700

    drm/i915: set IDICOS to medium uncore resources

Eugeni wants to benchmark the effect of this first.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
[danvet: added note]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We weren't disabling RC6 bits when bringing down RPS.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It should be working so let's turn it on by default and catch any possible
issues faster.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Just a cosmetic change to simplify the if statement.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Most of the RPS and RC6 enabling functionality is similar to what we had
on Gen6/Gen7, so we preserve most of the registers.

Note that Haswell only has RC6, so account for that as well. As suggested
by Daniel Vetter, to reduce the amount of changes in the patch, we still
write the RC6p/RC6pp thresholds, but those are ignored on Haswell.

Note: Some discussion about the nature of the new tuning constants
popped up in review - the answer is that we don't know why they've
changed, but the guide from VPG with the magic numbers simply has
different values now.

v2: Squash fix for ?: vs | operation precende bug into this patch.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
[danvet: Added note to commit message. Squashed fix.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Tidy up the routines for interacting with the GT (in particular the
forcewake dance) which are scattered throughout the code in a single
structure.

v2: use wait_for_atomic for polling.

v3: *really* use wait_for_atomic for polling.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The enable functions grabbed dev->struct_mutex themselves, whereas
the disable functions expected dev->struct_mutex to be held by the
caller. Move the locking out to the (currently only) callsite of
intel_enable_gt_powersave to make this more consistent.

Originally this was prep work for future patches, but I've chased down
a totally wrong alley. Still, I think this is a sensible
clarification.
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

... instead of calling each one for each generation indiviudally.

Notice that we've already managed to be inconsistent, the resume path
is missing an IS_VLV check. As a nice benefit we can mark all the
platform specific enable/disable functions as static and hide them in
intel_pm.c
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Enable the on-chip messaging between the display engine and the GT.

v2: use bit definitions for DPFLIPSTAT reg
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Prevents a possible hang: WaDisableL3Bank2xClockGate.

v2: only apply to VLV, IVB doesn't need this anymore

References: https://bugs.freedesktop.org/show_bug.cgi?id=50245Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Another required workaround for a potential hang:
WaDisableTDLUnitClockGating.

v2: only apply this to VLV, IVB doesn't need it anymore (Eugeni)

References: https://bugs.freedesktop.org/show_bug.cgi?id=50245Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

According to the bspec for MBCTL:

Driver must set bit in the following scenarios:
  - to realod teh h/w boot context every time it gets loaded through OS
  - after an FLR clears the register (BIOS won't run afterwards)

References: https://bugs.freedesktop.org/show_bug.cgi?id=50237Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The RCBP workaround still applies on these chips, and we need VDS as well.

v2: remove MB boot fetch that snuck in (Daniel)
    add workaround tags to comments for easier internal tracking (Daniel)
v3: only apply RCPB and VDS on SNB and VLV, IVB doesn't need them (Eugeni)

References: https://bugs.freedesktop.org/show_bug.cgi?id=50251Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

I'm seeing about a 5% FPS improvement across various benchmarks on my
IVB i3. Rumor has it that the higher end parts show even more benefit.

This derives from a patch originally given to me by Bernard. The docs
are  confusing about the definition names (ie. medium really seems like
max), but it would seem it gives more cache to the GT at the expense of
uncore. This configuration makes the split most in favor of the GT. I've
not tried the other IDICOS values.

Cc: "Kilarski, Bernard R" <bernard.r.kilarski@intel.com>
Acked-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

As we switch on/off the primary plane if it is completely obscured by an
overlapping video sprite, we also nee to make sure that we update the
FBC configuration at the same time.

v2: Not all crtcs are intel_crtcs, as spotted by Daniel.
v3: Boot testing rules.

References: https://bugs.freedesktop.org/show_bug.cgi?id=50238Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NJani Nikula <jani.nikula@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We still don't understand why this fails exactly, but if fails way
too often for a simple debug information. Furthermore the current
ducttape should prevent the gpu from getting stuck at low frequencies.

Hence tune down the dmesg noise.

Note that the known failure case is that the register read returns 0
when the gpu gets confused.

v2: Add comments about the known failure case.
Acked-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Media turbo requests can either use RPVSWREQ or RPNSWREQ to indicate
what the interrupt handler should do.  Since we only deal with the
latter in our turbo code, make the media engine use that for turbo
requests.

Cc: stable@vger.kernel.org.
Tested-by: NJoe Bloggsian <joebloggsian@gmail.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In order to avoid missed down-interrupts when coming out of RC6, it is
advised that we always reset the down-threshold upon a PM event. This is
due to that the PM unit goes through a little dance when coming out of
RC6, it first brings the GPU up at the lowest frequency then a short
time later it restores the thresholds. During that interval, the
down-interval may expire and the interrupt be suppressed.

Now aware of the dance taking place within the GPU when coming out of
RC6, one wonders what other writes need to be queued in the fifo buffer
in order to be properly sequenced; setting the RP state appears to be
one.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=44006Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Acked-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In many places we wish to iterate over the rings associated with the
GPU, so refactor them to use a common macro.

Along the way, there are a few code removals that should be side-effect
free and some rearrangement which should only have a cosmetic impact,
such as error-state.

Note that this slightly changes the semantics in the hangcheck code:
We now always cycle through all enabled rings instead of
short-circuiting the logic.

v2: Pull in a couple of suggestions from Ben and Daniel for
intel_ring_initialized() and not removing the warning (just moving them
to a new home, closer to the error).
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
[danvet: Added note to commit message about the small behaviour
change, suggested by Ben Widawsky.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The line time can be programmed according to the number of horizontal
pixels vs effective pixel rate ratio.

v2: improve comment as per Chris Wilson suggestion

v3: incorporate latest changes in specs.

v4: move into wm update routine, also mention that the same routine can
program IPS watermarks. We do not have their enablement code yet, nor
handle the required clock settings at the moment, so this patch won't
program those values for now.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

For now, we simple reuse the Ivy Bridge routines here.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This attempts to enable all the available power wells during the
initialization.

Those power wells can be enabled in parallel or on-demand, and disabled
when no longer needed, but this is out of scope of this initial
enablement. Proper tracking of who uses which power well will require
a considerable rework of our display handling, so we just leave them all
enabled when the driver is loaded for now.

v2: use more generic and future-proof code
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This adds proper support for calculating those watermarks, checking for
number of available pipes instead of specific GPU variants when deciding
if watermarks for 3rd pipe are necessary.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Every time we use the device after a period of idleness, check that the
power management setup is still sane. This is to workaround a bug
whereby it seems that we begin suppressing power management interrupts,
preventing SandyBridge+ from going into turbo mode.

This patch does have a side-effect. It removes the mark-busy for just
moving the cursor - we don't want to increase the render clock just for
the sprite, though we may want to bump the display frequency. I'd argue
that we do not, and certainly don't want to take the struct_mutex here
due to the large latencies that introduces.

References: https://bugs.freedesktop.org/show_bug.cgi?id=44006Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Because this is the place where we actually use the results of
them.
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We now have a nice home for power management code, so let's use it!

v2: Resolve conflict agains "Only enable IPS polling for gen5"
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The LP refers to 'low priority' as opposed to the high priority
ring on gen2/3. So lets constrain its use to the code of that era.

Unfortunately we can't yet completely remove the associated
macros from common headers and shove them into i915_dma.c to
the other dri1 legacy support code, a few cleanups are still
missing for that.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

I've missed this one.

v2: Chris Wilson noticed another register.
v3: Color choice improvements.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>