This commit moves force wake support routines into intel_pm modules, and
exports the gen6_gt_check_fifodbg routine (used in I915_READ).
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Acked-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

For Haswell, on some of the early hardware revisions, it is possible to
run into issues when RC6 state is enabled and when pipes change state.

v2: add comment saying that this is for early revisions only.

v3: beautify as suggested by Daniel Vetter.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Acked-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This is based on Ivy Bridge clock gating for now, but is subject to
changes in the future.

Note: Compared to the ivb clock gating this drops the the IDICOS
medium uncore sharing tuned in

commit 20848223
Author: Ben Widawsky <ben@bwidawsk.net>
Date:   Fri May 4 18:58:59 2012 -0700

    drm/i915: set IDICOS to medium uncore resources

Eugeni wants to benchmark the effect of this first.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
[danvet: added note]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We weren't disabling RC6 bits when bringing down RPS.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It should be working so let's turn it on by default and catch any possible
issues faster.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Just a cosmetic change to simplify the if statement.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Most of the RPS and RC6 enabling functionality is similar to what we had
on Gen6/Gen7, so we preserve most of the registers.

Note that Haswell only has RC6, so account for that as well. As suggested
by Daniel Vetter, to reduce the amount of changes in the patch, we still
write the RC6p/RC6pp thresholds, but those are ignored on Haswell.

Note: Some discussion about the nature of the new tuning constants
popped up in review - the answer is that we don't know why they've
changed, but the guide from VPG with the magic numbers simply has
different values now.

v2: Squash fix for ?: vs | operation precende bug into this patch.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
[danvet: Added note to commit message. Squashed fix.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Tidy up the routines for interacting with the GT (in particular the
forcewake dance) which are scattered throughout the code in a single
structure.

v2: use wait_for_atomic for polling.

v3: *really* use wait_for_atomic for polling.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The enable functions grabbed dev->struct_mutex themselves, whereas
the disable functions expected dev->struct_mutex to be held by the
caller. Move the locking out to the (currently only) callsite of
intel_enable_gt_powersave to make this more consistent.

Originally this was prep work for future patches, but I've chased down
a totally wrong alley. Still, I think this is a sensible
clarification.
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

... instead of calling each one for each generation indiviudally.

Notice that we've already managed to be inconsistent, the resume path
is missing an IS_VLV check. As a nice benefit we can mark all the
platform specific enable/disable functions as static and hide them in
intel_pm.c
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Enable the on-chip messaging between the display engine and the GT.

v2: use bit definitions for DPFLIPSTAT reg
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Prevents a possible hang: WaDisableL3Bank2xClockGate.

v2: only apply to VLV, IVB doesn't need this anymore

References: https://bugs.freedesktop.org/show_bug.cgi?id=50245Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Another required workaround for a potential hang:
WaDisableTDLUnitClockGating.

v2: only apply this to VLV, IVB doesn't need it anymore (Eugeni)

References: https://bugs.freedesktop.org/show_bug.cgi?id=50245Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

According to the bspec for MBCTL:

Driver must set bit in the following scenarios:
  - to realod teh h/w boot context every time it gets loaded through OS
  - after an FLR clears the register (BIOS won't run afterwards)

References: https://bugs.freedesktop.org/show_bug.cgi?id=50237Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The RCBP workaround still applies on these chips, and we need VDS as well.

v2: remove MB boot fetch that snuck in (Daniel)
    add workaround tags to comments for easier internal tracking (Daniel)
v3: only apply RCPB and VDS on SNB and VLV, IVB doesn't need them (Eugeni)

References: https://bugs.freedesktop.org/show_bug.cgi?id=50251Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

I'm seeing about a 5% FPS improvement across various benchmarks on my
IVB i3. Rumor has it that the higher end parts show even more benefit.

This derives from a patch originally given to me by Bernard. The docs
are  confusing about the definition names (ie. medium really seems like
max), but it would seem it gives more cache to the GT at the expense of
uncore. This configuration makes the split most in favor of the GT. I've
not tried the other IDICOS values.

Cc: "Kilarski, Bernard R" <bernard.r.kilarski@intel.com>
Acked-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

As we switch on/off the primary plane if it is completely obscured by an
overlapping video sprite, we also nee to make sure that we update the
FBC configuration at the same time.

v2: Not all crtcs are intel_crtcs, as spotted by Daniel.
v3: Boot testing rules.

References: https://bugs.freedesktop.org/show_bug.cgi?id=50238Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NJani Nikula <jani.nikula@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We still don't understand why this fails exactly, but if fails way
too often for a simple debug information. Furthermore the current
ducttape should prevent the gpu from getting stuck at low frequencies.

Hence tune down the dmesg noise.

Note that the known failure case is that the register read returns 0
when the gpu gets confused.

v2: Add comments about the known failure case.
Acked-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Media turbo requests can either use RPVSWREQ or RPNSWREQ to indicate
what the interrupt handler should do.  Since we only deal with the
latter in our turbo code, make the media engine use that for turbo
requests.

Cc: stable@vger.kernel.org.
Tested-by: NJoe Bloggsian <joebloggsian@gmail.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In order to avoid missed down-interrupts when coming out of RC6, it is
advised that we always reset the down-threshold upon a PM event. This is
due to that the PM unit goes through a little dance when coming out of
RC6, it first brings the GPU up at the lowest frequency then a short
time later it restores the thresholds. During that interval, the
down-interval may expire and the interrupt be suppressed.

Now aware of the dance taking place within the GPU when coming out of
RC6, one wonders what other writes need to be queued in the fifo buffer
in order to be properly sequenced; setting the RP state appears to be
one.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=44006Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Acked-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In many places we wish to iterate over the rings associated with the
GPU, so refactor them to use a common macro.

Along the way, there are a few code removals that should be side-effect
free and some rearrangement which should only have a cosmetic impact,
such as error-state.

Note that this slightly changes the semantics in the hangcheck code:
We now always cycle through all enabled rings instead of
short-circuiting the logic.

v2: Pull in a couple of suggestions from Ben and Daniel for
intel_ring_initialized() and not removing the warning (just moving them
to a new home, closer to the error).
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
[danvet: Added note to commit message about the small behaviour
change, suggested by Ben Widawsky.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The line time can be programmed according to the number of horizontal
pixels vs effective pixel rate ratio.

v2: improve comment as per Chris Wilson suggestion

v3: incorporate latest changes in specs.

v4: move into wm update routine, also mention that the same routine can
program IPS watermarks. We do not have their enablement code yet, nor
handle the required clock settings at the moment, so this patch won't
program those values for now.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

For now, we simple reuse the Ivy Bridge routines here.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This attempts to enable all the available power wells during the
initialization.

Those power wells can be enabled in parallel or on-demand, and disabled
when no longer needed, but this is out of scope of this initial
enablement. Proper tracking of who uses which power well will require
a considerable rework of our display handling, so we just leave them all
enabled when the driver is loaded for now.

v2: use more generic and future-proof code
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This adds proper support for calculating those watermarks, checking for
number of available pipes instead of specific GPU variants when deciding
if watermarks for 3rd pipe are necessary.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Every time we use the device after a period of idleness, check that the
power management setup is still sane. This is to workaround a bug
whereby it seems that we begin suppressing power management interrupts,
preventing SandyBridge+ from going into turbo mode.

This patch does have a side-effect. It removes the mark-busy for just
moving the cursor - we don't want to increase the render clock just for
the sprite, though we may want to bump the display frequency. I'd argue
that we do not, and certainly don't want to take the struct_mutex here
due to the large latencies that introduces.

References: https://bugs.freedesktop.org/show_bug.cgi?id=44006Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Because this is the place where we actually use the results of
them.
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We now have a nice home for power management code, so let's use it!

v2: Resolve conflict agains "Only enable IPS polling for gen5"
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The LP refers to 'low priority' as opposed to the high priority
ring on gen2/3. So lets constrain its use to the code of that era.

Unfortunately we can't yet completely remove the associated
macros from common headers and shove them into i915_dma.c to
the other dri1 legacy support code, a few cleanups are still
missing for that.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

I've missed this one.

v2: Chris Wilson noticed another register.
v3: Color choice improvements.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The specs recommend that this bit be set on PineView. No reason is
given, but it sounds like a powersaving bit that we should expect the
BIOS to be setting...

v2: Rebase on top of _MASKED_ENABLE_BIT
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Copy&pasted from the vlv setup code. According to docs, we need that
on ivb, too.

v2: Use new masked bit handling macros.
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

... and put them to so good use.

Note that there's functional change in vlv clock gating code, we now
no longer spuriously read back the current value of the bit. According
to Bspec the high bits should always read zero, so ORing this in
should have no effect.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This got lost in the intel_pm.c move.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This adds intel_pm routine for generic power-related infrastructure
initialization.

v2: now that all the platform-specific stuff is initialized in one place, we
can also add back the static definitions to platform-specific functions which
we abstract now.
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Acked-by: NBen Widawsky <benjamin.widawsky@intel.com>
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This moves the clock gating-related functions into intel_pm module.

Also, please note that we do change the function type from static to
non-static in this patch for the move, to prevent breaking bisecting with
non-working intermediate commit. Those are returned back to static form in
the following patch which setups a generic PM initialization function,
which was split into a different one to simplify review.

v2: rebase on top of latest drm-intel-next-queued to incorporate all the
changes that went there meanwhile.
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Acked-by: NBen Widawsky <benjamin.widawsky@intel.com>
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This moves the Ironlake energy monitoring functionality into intel_pm
module.
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Acked-by: NBen Widawsky <benjamin.widawsky@intel.com>
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This moves DRPS, RPS and RC6-related functionality into intel_pm module.

It also removes the linux/cpufreq.h include from intel_display, as its
only user was the GPU turbo-related functionality in Gen6+ code path.

v2: rebase on top of latest drm-intel-next-queued adding the bits that
shifted around since the last patch.
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Acked-by: NBen Widawsky <benjamin.widawsky@intel.com>
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The previous patch had way too long lines, this fixes them to fit into a
reasonable screen space.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Move watermarks and helper functions (such as cxsr and fifo buffers) into
intel_pm module.
Signed-off-by: NEugeni Dodonov <eugeni.dodonov@intel.com>
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>