GEN8_L3SQCREG1 isn't saved in the context (verified by going through
a context dump), and so we shouldn't be using the ring w/a code to
initialize it. Also Bspec explicitly talks about MMIO and writing it
with the CPU.

Additionally there's another w/a WaTempDisableDOPClkGating:bdw which
tells us to disable DOP clock gating around the GEN8_L3SQCREG1 write
to make sure everyone notices the change. So let's do that as well.

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We're not using ilk_init_lp_watermarks() on BDW for some reason.
Probably due to the BDW patches and the relevant WM patches landing
roughlly at the same time. Fix it up.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

If the client stalls on a congested request, chosen to be 20ms old to
match throttling, allow the client a free RPS boost.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: s/rq/req/]
[danvet: s/0/NULL/ reported by 0-day build]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Now that we have internal clients, rather than faking a whole
drm_i915_file_private just for tracking RPS boosts, create a new struct
intel_rps_client and pass it along when waiting.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: s/rq/req/]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Since we will often pageflip to an active surface, we will often have to
wait for the surface to be written before issuing the flip. Also we are
likely to wait on that surface in plenty of time before the vblank.
Since we have a mechanism for boosting when a flip misses the expected
vblank, curtain the number of times we RPS boost when simply waiting for
mmioflip.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: s/rq/req/]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Ring switches can occur many times per frame, and are often out of
control, causing frequent RPS boosting for no practical benefit. Treat
the sw semaphore synchronisation as a separate client and only allow it
to boost once per busy/idle cycle.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: s/rq/req/]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The merged seqno->request conversion from John called request
variables req, but some (not all) of Chris' recent patches changed
those to just rq. We've had a lenghty (and inconclusive) discussion on
irc which is the more meaningful name with maybe at most a slight bias
towards req.

Given that the "don't change names without good reason to avoid
conflicts" rule applies, so lets go back to a req everywhere for
consistency. I'll sed any patches for which this will cause conflicts
before applying.

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: John Harrison <John.C.Harrison@Intel.com>
[danvet: s/origina/merged/ as pointed out by Chris - the first
mass-conversion patch was from Chris, the merged one from John.]
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>

After feedback from the hardware team, now we set the GPU min/idel freq to RPe.
Punit is expecting us to operate GPU between Rpe & Rp0. If we drop the
frequency to RPn, punit is failing to change the vgg input voltage to
minimum :(

Since Punit validates the rps range [RPe, RP0]. This patch
removes unused cherryview_rps_min_freq function.

v2: Change commit message

v3: set min_freq before idle_freq (chris)

v4: Squash 'Remove unused rps min function' patch
Signed-off-by: NDeepak S <deepak.s@linux.intel.com>
Acked-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It is observed on BSW that requesting a new frequency from Punit
does nothing when the GPU is in rc6, and if we let GPU enter rc6
with a high frequency, Vnn remains slightly higher than at minimum
frequency. Extending vlv_set_rps_idle() workaround on CHV/BSW.

v2: Update commit msg (Ville)
suggested-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDeepak S <deepak.s@linux.intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

When GPU is idle on VLV, Request freq to punit should be good enough to
get the voltage back to VNN. Also, make sure gfx clock force applies
before requesting the freq fot vlv.

v2: Do forcewake before setting idle frequency (ville)
    Update function comments to match the code (Deepak)

v3: Fix get/put across idle frequency Request. (Ville)

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=75244suggested-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDeepak S <deepak.s@linux.intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

As we're doing throughout the code, being optimistic that platform n + 1
will mostly reuse the same things as platform n allows us to minimize
the enabling work needed.

This time, it's about the number of WM levels.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Skylake nv12 format requires dbuf (aka. ddb) calculations
and programming for each of y and uv sub-planes. Made minor
changes to reuse current dbuf calculations and programming
for uv plane. i.e., with this change, existing computation
is used for either packed format or uv portion of nv12
depending on incoming format. Added new code for dbuf
computation and programming for y plane.

This patch is a pre-requisite for adding NV12 format support.
Actual nv12 support is coming in later patches.
Signed-off-by: NChandra Konduru <chandra.konduru@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Robert noticed that the FF_SLICE_CS_CHICKEN2 offset was wrong. Ooops.

Ville noticed that the write was wrong since FF_SLICE_CS_CHICKEN2 is a
masked register. Re-oops.

A wonder if went through 2 people while having roughly a bug per line...

The problem was introduced in the original patch:

  commit 2caa3b26
  Author: Damien Lespiau <damien.lespiau@intel.com>
  Date:   Mon Feb 9 19:33:20 2015 +0000

      drm/i915/skl: Implement WaDisableChickenBitTSGBarrierAckForFFSliceCS

v2: Also fix the register write (Ville)
Reported-by: NRobert Beckett <robert.beckett@intel.com>
Reported-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Cc: Robert Beckett <robert.beckett@intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Nick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Replace the hardcoded 9 with a call to intel_freq_opcode(450).
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Based on the spec, Setting up static BIAS for GPU to improve the
rps performace.

v2: rename reg defn to match spec. (Ville)

v3: Updated bias setting for chv (Deepak)
Signed-off-by: NDeepak S <deepak.s@linux.intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

After feedback from the hardware team we are changing the RC6
promotional timer to increase the power saving without
changing performance.
Signed-off-by: NDeepak S <deepak.s@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Set TLBPF in TILECTL. This fixes an issue with BXT HW seeing
corrupted pte entries.

v2:
- move the workaround to bxt_init_clock_gating (imre)

Signed-off-by: Robert Beckett <robert.beckett@intel.com> (v1)
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2:
- Make the condition to select between SKL and BXT consistent with the
  corresponding condition in init_workarounds_ring (Nick)
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

When RC6 along with Render power gating is enabled, GPU hang
happens due to lack of synchronization between GTI and Render
power gating.

v2: Updated commit message and WA name (Damien)

Change-Id: If1614206341eb52a21eadae8c5ebb2655029b50c
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NSagar Kamble <sagar.a.kamble@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Change-Id: I4253459c075c50d9b6f034b4ed4ad2f54cd7d1d7
Signed-off-by: NSagar Kamble <sagar.a.kamble@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

With boosting for missed pageflips, we have a much stronger indication
of when we need to (temporarily) boost GPU frequency to ensure smooth
delivery of frames. So now only allow each client to perform one RPS boost
in each period of GPU activity due to stalling on results.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Deepak S <deepak.s@linux.intel.com>
Reviewed-by: NDeepak S <deepak.s@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

If we hit a vblank and see that have a pageflip queue but not yet
processed, ensure that the GPU is running at maximum in order to clear
the backlog. Pageflips are only queued for the following vblank, if we
miss it, there will be a visible stutter. Boosting the GPU frequency
doesn't prevent us from missing the target vblank, but it should help
the subsequent frames hitting theirs.

v2: Reorder vblank vs flip-complete so that we only check for a missed
flip after processing the completion events, and avoid spurious boosts.

v3: Rename missed_vblank
v4: Rebase
v5: Cancel the outstanding work in runtime suspend
v6: Rebase
v7: Rebase required fixing
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Deepak S<deepak.s@linux.intel.com>
Reviewed-by: Deepak S<deepak.s@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Reuse the same reclocking strategy for Baytail as on its bigger brethren,
Sandybridge and Ivybridge. In particular, this makes the device quicker
to reclock (both up and down) though the tendency now is to downclock
more aggressively to compensate for the RPS boosts.

v2: Rebase
v3: Exclude Cherrytrail as Deepak was concerned that the increased
number of register writes would wake the common powerwell too often.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Deepak S <deepak.s@linux.intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NDeepak S <deepak.s@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Both WaDisableSDEUnitClockGating and WaSetGAPSunitClckGateDisable are
needed on B0 as well.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

../drivers/gpu/drm/i915/intel_pm.c:3185:45: warning: Initializer entry defined twice
../drivers/gpu/drm/i915/intel_pm.c:3185:52:   also defined here
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NAntti Koskipää <antti.koskipaa@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Unify the HSW/BDW/SKL cdclk extraction code to conform to the same
.get_display_clock_speed() mold that all the other platforms
use.

v2: Update due to SKL code getting added

v3: Rebase on top of -nightly (introduction of intel_audio.c) (Mika Kahola)
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMika Kahola <mika.kahola@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
[danvet: Add v3 note as suggested by Damien.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

drivers/gpu/drm/i915/intel_pm.c:2913:4-5: Unneeded semicolon

 Removes unneeded semicolon.

Generated by: scripts/coccinelle/misc/semicolon.cocci

CC: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NFengguang Wu <fengguang.wu@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2: Pass in rotation info to sprite plane updates as well.

v3: Use helper to determine 90/270 rotation. (Michel Thierry)

v4: Rebased for fb modifiers and atomic changes.

For: VIZ-4546
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: Michel Thierry <michel.thierry@intel.com> (v3)
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Use both up/down manual ei calcuations for symmetry and greater
flexibility for reclocking, instead of faking the down interrupt based
on a fixed integer number of up interrupts.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Deepak S<deepak.s@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Rewrite commit 31685c25
Author: Deepak S <deepak.s@linux.intel.com>
Date:   Thu Jul 3 17:33:01 2014 -0400

    drm/i915/vlv: WA for Turbo and RC6 to work together.

Other than code clarity, the major improvement is to disable the extra
interrupts generated when idle.  However, the reclocking remains rather
slow under the new manual regime, in particular it fails to downclock as
quickly as desired. The second major improvement is that for certain
workloads, like games, we need to combine render+media activity counters
as the work of displaying the frame is split across the engines and both
need to be taken into account when deciding the global GPU frequency as
memory cycles are shared.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Deepak S <deepak.s@linux.intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Deepak S<deepak.s@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

When we idle, we set the GPU frequency to the hardware minimum (not user
minimum). We introduce a new variable to distinguish between the
different roles, and to allow easy tuning of the idle frequency without
impacting over aspects of RPS. Setting the minimum frequency should be a
safety blanket as the pcu on the GPU should be power gating itself
anyway. However, in order for us to do set the absolute minimum
frequency, we need to relax a few of our assertions that we do not
exceed the user limits.

v2: Add idle_freq
v3: Init idle_freq for vlv and add a bunch of WARNs
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Deepak S <deepak.s@linux.intel.com>
Reviewed-by: Deepak S<deepak.s@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Enable the RPS interrupts programming(enable/disable/reset) for GEN9,
as missing changes to enable the RPS support on GEN9 have been added.
Signed-off-by: NAkash Goel <akash.goel@intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

On SKL, GT frequency is programmed in units of 16.66 MHZ units compared
to 50 MHZ for older platforms. Also the time value specified for Up/Down EI &
Up/Down thresholds are expressed in units of 1.33 us, compared to 1.28
us for older platforms. So updated the gen9_enable_rps function as per that.

v2: Updated to use new macro GT_INTERVAL_FROM_US

v3: Removed the initial setup of certain registers, from gen9_enable_rps,
    which gets overridden later from gen6_set_rps (Damien)

v4: Removed the enabling of rps interrupts, from gen9_enable_rps.
    To be done from intel_gen6_powersave_work only, as done for other
    platforms also.
Signed-off-by: NAkash Goel <akash.goel@intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

RP Interrupt Up/Down Frequency Limits register (A014) definition
has changed for SKL. Updated the gen6_rps_limits function as per that

v2: Renamed the function to intel_rps_limits (Chris)
Signed-off-by: NAkash Goel <akash.goel@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Prior to SKL, the time period programmed in Up/Down EI & Up/Down
threshold registers was in units of 1.28 micro seconds. But for
SKL, the units have changed (1.333 micro seconds).
Have generalized the implementation of gen6_set_rps_thresholds function,
by removing the hard coding done in it as per 1.28 micro seconds.

v2: Renamed the local variables & removed superfluous comments (Chris)
Signed-off-by: NAkash Goel <akash.goel@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

On SKL, the frequency is programmed differently in RPNSWREQ (A008)
register (from bits 23 to 31, compared to bits 24 to 31). So updated
the gen6_set_rps function, as per this change.
Signed-off-by: NAkash Goel <akash.goel@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>