Currently we've only frobbed this bit at irq_init time, but did
not restore it at resume time. Move it to the gen3 clock gating
function to fix this.

Notice while reading through code.

Cc: stable@vger.kernel.org (for 3.5 only)
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

James Bottomley reported [1] a massive power regression, due to the
enabling of semaphores by default in 3.5. A workaround for him is to
again disable semaphores. And indeed, his system has a very hard time
to enter rc6 with semaphores enabled.

Ben Widawsky run around with a kill-a-watt a lot and noticed:
- There are indeed a few rare systems that seem to have a hard time
  entering rc6 when desktop-idle.
- One machine, The Indestructible Toshiba regressed in this behaviour
  between 3.5 and 3.6 in a merge commit! So rc6 behaviour with the
  current setting seems to be highly timing dependent and not robust
  at all.
- The behaviour James reported wrt semaphores seems to be a freak
  timing thing that only happens on his specific machine, confirming
  that enabling semaphores shouldn't reduce rc6 residency.

Now furthermore the Google ChromeOS guys reported [2] a while ago that
at least on some machines a simply a blinking cursor can keep the gpu
turbo at the highest frequency. This is because the current rps limits
used on snb/ivb are highly asymmetric.

On the theory that gpu turbo and rc6 tuning values are related, we've
tried whether the much saner looking (since much less asymmetric) rps
tuning values used for hsw would also help entering rc6 more robustly.

And it seems to mostly work, and we don't really have the resources to
through-roughly tune things in any better way: The values from the
ChromeOS ppl seem to fare a bit worse for James' machine, so I guess
we better stick with something vpg (the gpu hw/windows group)
provided, hoping that they've done their jobs.

Reference[1]: http://lists.freedesktop.org/archives/dri-devel/2012-July/025675.html
Reference[2]: http://lists.freedesktop.org/archives/intel-gfx/2012-July/018692.html
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=53393Tested-by: NBen Widawsky <ben@bwidawsk.net>
Cc: stable@vger.kernel.org
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

... by adding seemingly redudant posting reads.

This little dragon lair exploded the first time around when we've
refactored the code a bit to use the common wait_for_atomic_us in
"drm/i915: Group the GT routines together in both code and vtable",
which caused QA to file fdo bug #51738.

Chris Wilson entertained a few approaches to fixing #51738: Replacing
the udelay(1) with the previously-used udelay(10) (or any other
"sufficiently larger" delay), adding a posting read, or ditching the
delay completely and using cpu_relax. We went with the cpu_relax and
"915: Workaround hang with BSD and forcewake on SandyBridge". Which
blew up in fdo bug #52424, but adding the posting read while still
using cpu_relax seems to also fix that, it looks like the
posting read is the important ingriedient to fix these rc6 related
hangs on snb.

Popular theories as to why this is like it is include:
- A herd of pink elephants got royally angered somehow.

- The gpu has internally different functional units and judging by the
  register offsets, the forcewake request register and the forcewake
  ack registers are _not_ in the same functional unit (or at least
  aren't reached through the same routes). Hence the posting read
  syncs up with the wrong block and gets the entire gpu confused.

- ...

As a minimal ducttape fix for 3.6, let's just put these posting reads
into place again. We can try fancier approaches (like adding back the
cpu_relax instead of the udelay) in -next.

This (re-)fixes a regression introduced in

commit 990bbdad
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date:   Mon Jul 2 11:51:02 2012 -0300

    drm/i915: Group the GT routines together in both code and vtable

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Tested-by: NChris Wilson <chris@chris-wilson.co.uk>
Tested-by: NDu Yan <yanx.du@intel.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=52424
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=51738uSigned-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This initializes power wells within the modeset_init_hw routine.
Testing has shown that this works for both driver load time and for
suspend-resume code paths.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

There is little point waking up every 10ms to service an interrupt which
we then promptly ignore. So only program the the PMIER to enable
interrupts for those events which we do handle, not all of them!
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Cc: Eugeni Dodonov <eugeni.dodonov@intel.com>
Cc: Ben Widawsky <ben@bwidawsk.net>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This pollutes dmesg output even if we do not have FBC for the device, so
move the DRM_DEBUG_KMS statement lower.

v2: just kill the message as suggested by Daniel.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

While creating the new enable/disable_gt_powersave functions in

commit 8090c6b9
Author: Daniel Vetter <daniel.vetter@ffwll.ch>
Date:   Sun Jun 24 16:42:32 2012 +0200

    drm/i915: wrap up gt powersave enabling functions

I've botched up the handling of ironlake_disable_rc6. Fix this up by
calling it at the right place. Note though that ironlake_disable_rc6
does a bit more than just disabling rc6 - it also tears down all the
allocated context objects.

Hence we need to move intel_teardown_rc6 out and directly call it from
intel_modeset_cleanup.

Also properly mark ironlake_enable_rc6 as static and kill the un-used
declaration in i915_drv.h.

Note: In review a question popped out why disable_rc6 also tears down
the backing object and why we should move that out - it's simply for
consistency with gen6+ rps code, which does it that way.

Cc: Ben Widawsky <ben@bwidawsk.net>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This commit moves force wake support routines into intel_pm modules, and
exports the gen6_gt_check_fifodbg routine (used in I915_READ).
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Acked-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

For Haswell, on some of the early hardware revisions, it is possible to
run into issues when RC6 state is enabled and when pipes change state.

v2: add comment saying that this is for early revisions only.

v3: beautify as suggested by Daniel Vetter.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Acked-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This is based on Ivy Bridge clock gating for now, but is subject to
changes in the future.

Note: Compared to the ivb clock gating this drops the the IDICOS
medium uncore sharing tuned in

commit 20848223
Author: Ben Widawsky <ben@bwidawsk.net>
Date:   Fri May 4 18:58:59 2012 -0700

    drm/i915: set IDICOS to medium uncore resources

Eugeni wants to benchmark the effect of this first.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
[danvet: added note]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We weren't disabling RC6 bits when bringing down RPS.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It should be working so let's turn it on by default and catch any possible
issues faster.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Just a cosmetic change to simplify the if statement.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Most of the RPS and RC6 enabling functionality is similar to what we had
on Gen6/Gen7, so we preserve most of the registers.

Note that Haswell only has RC6, so account for that as well. As suggested
by Daniel Vetter, to reduce the amount of changes in the patch, we still
write the RC6p/RC6pp thresholds, but those are ignored on Haswell.

Note: Some discussion about the nature of the new tuning constants
popped up in review - the answer is that we don't know why they've
changed, but the guide from VPG with the magic numbers simply has
different values now.

v2: Squash fix for ?: vs | operation precende bug into this patch.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
[danvet: Added note to commit message. Squashed fix.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Tidy up the routines for interacting with the GT (in particular the
forcewake dance) which are scattered throughout the code in a single
structure.

v2: use wait_for_atomic for polling.

v3: *really* use wait_for_atomic for polling.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The enable functions grabbed dev->struct_mutex themselves, whereas
the disable functions expected dev->struct_mutex to be held by the
caller. Move the locking out to the (currently only) callsite of
intel_enable_gt_powersave to make this more consistent.

Originally this was prep work for future patches, but I've chased down
a totally wrong alley. Still, I think this is a sensible
clarification.
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

... instead of calling each one for each generation indiviudally.

Notice that we've already managed to be inconsistent, the resume path
is missing an IS_VLV check. As a nice benefit we can mark all the
platform specific enable/disable functions as static and hide them in
intel_pm.c
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Enable the on-chip messaging between the display engine and the GT.

v2: use bit definitions for DPFLIPSTAT reg
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Prevents a possible hang: WaDisableL3Bank2xClockGate.

v2: only apply to VLV, IVB doesn't need this anymore

References: https://bugs.freedesktop.org/show_bug.cgi?id=50245Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Another required workaround for a potential hang:
WaDisableTDLUnitClockGating.

v2: only apply this to VLV, IVB doesn't need it anymore (Eugeni)

References: https://bugs.freedesktop.org/show_bug.cgi?id=50245Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

According to the bspec for MBCTL:

Driver must set bit in the following scenarios:
  - to realod teh h/w boot context every time it gets loaded through OS
  - after an FLR clears the register (BIOS won't run afterwards)

References: https://bugs.freedesktop.org/show_bug.cgi?id=50237Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The RCBP workaround still applies on these chips, and we need VDS as well.

v2: remove MB boot fetch that snuck in (Daniel)
    add workaround tags to comments for easier internal tracking (Daniel)
v3: only apply RCPB and VDS on SNB and VLV, IVB doesn't need them (Eugeni)

References: https://bugs.freedesktop.org/show_bug.cgi?id=50251Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

I'm seeing about a 5% FPS improvement across various benchmarks on my
IVB i3. Rumor has it that the higher end parts show even more benefit.

This derives from a patch originally given to me by Bernard. The docs
are  confusing about the definition names (ie. medium really seems like
max), but it would seem it gives more cache to the GT at the expense of
uncore. This configuration makes the split most in favor of the GT. I've
not tried the other IDICOS values.

Cc: "Kilarski, Bernard R" <bernard.r.kilarski@intel.com>
Acked-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

As we switch on/off the primary plane if it is completely obscured by an
overlapping video sprite, we also nee to make sure that we update the
FBC configuration at the same time.

v2: Not all crtcs are intel_crtcs, as spotted by Daniel.
v3: Boot testing rules.

References: https://bugs.freedesktop.org/show_bug.cgi?id=50238Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NJani Nikula <jani.nikula@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We still don't understand why this fails exactly, but if fails way
too often for a simple debug information. Furthermore the current
ducttape should prevent the gpu from getting stuck at low frequencies.

Hence tune down the dmesg noise.

Note that the known failure case is that the register read returns 0
when the gpu gets confused.

v2: Add comments about the known failure case.
Acked-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Media turbo requests can either use RPVSWREQ or RPNSWREQ to indicate
what the interrupt handler should do.  Since we only deal with the
latter in our turbo code, make the media engine use that for turbo
requests.

Cc: stable@vger.kernel.org.
Tested-by: NJoe Bloggsian <joebloggsian@gmail.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In order to avoid missed down-interrupts when coming out of RC6, it is
advised that we always reset the down-threshold upon a PM event. This is
due to that the PM unit goes through a little dance when coming out of
RC6, it first brings the GPU up at the lowest frequency then a short
time later it restores the thresholds. During that interval, the
down-interval may expire and the interrupt be suppressed.

Now aware of the dance taking place within the GPU when coming out of
RC6, one wonders what other writes need to be queued in the fifo buffer
in order to be properly sequenced; setting the RP state appears to be
one.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=44006Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Acked-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In many places we wish to iterate over the rings associated with the
GPU, so refactor them to use a common macro.

Along the way, there are a few code removals that should be side-effect
free and some rearrangement which should only have a cosmetic impact,
such as error-state.

Note that this slightly changes the semantics in the hangcheck code:
We now always cycle through all enabled rings instead of
short-circuiting the logic.

v2: Pull in a couple of suggestions from Ben and Daniel for
intel_ring_initialized() and not removing the warning (just moving them
to a new home, closer to the error).
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
[danvet: Added note to commit message about the small behaviour
change, suggested by Ben Widawsky.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The line time can be programmed according to the number of horizontal
pixels vs effective pixel rate ratio.

v2: improve comment as per Chris Wilson suggestion

v3: incorporate latest changes in specs.

v4: move into wm update routine, also mention that the same routine can
program IPS watermarks. We do not have their enablement code yet, nor
handle the required clock settings at the moment, so this patch won't
program those values for now.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

For now, we simple reuse the Ivy Bridge routines here.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This attempts to enable all the available power wells during the
initialization.

Those power wells can be enabled in parallel or on-demand, and disabled
when no longer needed, but this is out of scope of this initial
enablement. Proper tracking of who uses which power well will require
a considerable rework of our display handling, so we just leave them all
enabled when the driver is loaded for now.

v2: use more generic and future-proof code
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This adds proper support for calculating those watermarks, checking for
number of available pipes instead of specific GPU variants when deciding
if watermarks for 3rd pipe are necessary.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Every time we use the device after a period of idleness, check that the
power management setup is still sane. This is to workaround a bug
whereby it seems that we begin suppressing power management interrupts,
preventing SandyBridge+ from going into turbo mode.

This patch does have a side-effect. It removes the mark-busy for just
moving the cursor - we don't want to increase the render clock just for
the sprite, though we may want to bump the display frequency. I'd argue
that we do not, and certainly don't want to take the struct_mutex here
due to the large latencies that introduces.

References: https://bugs.freedesktop.org/show_bug.cgi?id=44006Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Because this is the place where we actually use the results of
them.
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We now have a nice home for power management code, so let's use it!

v2: Resolve conflict agains "Only enable IPS polling for gen5"
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The LP refers to 'low priority' as opposed to the high priority
ring on gen2/3. So lets constrain its use to the code of that era.

Unfortunately we can't yet completely remove the associated
macros from common headers and shove them into i915_dma.c to
the other dri1 legacy support code, a few cleanups are still
missing for that.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

I've missed this one.

v2: Chris Wilson noticed another register.
v3: Color choice improvements.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The specs recommend that this bit be set on PineView. No reason is
given, but it sounds like a powersaving bit that we should expect the
BIOS to be setting...

v2: Rebase on top of _MASKED_ENABLE_BIT
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Copy&pasted from the vlv setup code. According to docs, we need that
on ivb, too.

v2: Use new masked bit handling macros.
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

... and put them to so good use.

Note that there's functional change in vlv clock gating code, we now
no longer spuriously read back the current value of the bit. According
to Bspec the high bits should always read zero, so ORing this in
should have no effect.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>