Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Don't rely on fb->bits_per_pixel as intel_framebuffer_init is not
filling bits_per_pixel field of fb-struct for YUV pixel format.
This leads to divide by zero error during watermark calculation.
Signed-off-by: NKumar, Mahesh <mahesh1.kumar@intel.com>
Cc: Konduru, Chandra <chandra.konduru@intel.com>
Reviewed-by: NSonika Jindal <sonika.jindal@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Make LPT:LP checks look neater by wrapping the details in a
new HAS_PCH_LPT_LP() macro.
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Cc: Ben Widawsky <benjamin.widawsky@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Signed-off-by: NArun Siluvery <arun.siluvery@linux.intel.com>
Tested-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=90854Tested-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

No need to pass the planelist when everyone just uses
dev->mode_config.plane_list anyway.

I want to add a pile more of iterators with unified (obj, dev)
arguments. This is just prep.
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>

This reverts commit 6adfb1ef.

Ironlake RPS code runs under an irqsave spinlock and hence sleeping
isn't allowed. Not a this long delay while blocking irqs isn't great
at all, but fixing the locking scheme is a lot more involved.

So just revert for now.

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reported-by: Nkernel test robot <ying.huang@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>

Fix divide by zero if we end up updating the watermarks
with zero dotclock.

This is a stop gap measure to allow module load in cases
where our state keeping fails.

v2: WARN_ON added (Paulo)

Cc: Paulo Zanoni <przanoni@gmail.com>
Cc: Damien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In Indirect context w/a batch buffer,
+WaFlushCoherentL3CacheLinesAtContextSwitch:skl,bxt

v2: address static checker warning where unsigned value was checked for
less than zero which is never true (Dan Carpenter).

v3: The WA uses default value of GEN8_L3SQCREG4 during flush but that disables
some other WA; update default value to retain it and document dependency (Mika).

Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Signed-off-by: NArun Siluvery <arun.siluvery@linux.intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2: Patch leakage fixed
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Added a new HAS_CORE_RING_FREQ macro, currently used in
gen6_update_ring_freq & i915_ring_freq_table debugfs function.
The programming & read of ring frequency table is needed for newer
GEN(>=6) platforms, except VLV/CHV.

Issue: VIZ-5144
Suggested-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NAkash Goel <akash.goel@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Ring frequency table programming is not required on BXT. Added separate
checks to enable the programming only for SKL & skip for BXT.

v2: Removed the BXT check from gen6_update_ring_freq function

Issue: VIZ-5144
Signed-off-by: NAkash Goel <akash.goel@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi at intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently we update the freq before masking the interrupts, which can
allow new interrupts to occur before the frequency has changed. These
extra interrupts might waste some cpu cycles. This patch corrects
this by masking interrupts prior to updating the frequency.

Note from Chris:
"Well it won't waste CPU cycles as the interrupt is also masked by the
threshold limits, but there should be no harm at all in reordering the
patch so, and it does make a certain amount of sense."
Signed-off-by: NDeepak S <deepak.s@intel.com>
Signed-off-by: NPraveen Paneri <praveen.paneri@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: Add note from Chris.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Ring frequency table programming changes for SKL. No need for a
floor on ring frequency, as the issue of performance impact with
ring running below DDR frequency, is believed to be fixed on SKL

v2: Removed the check for avoiding ring frequency programming for BXT (Rodrigo)

Issue: VIZ-5144
Signed-off-by: NAkash Goel <akash.goel@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Read the efficient frequency (aka RPe) value through the the mailbox
command (0x1A) from the pcode, as done on Haswell and Broadwell.
The turbo minimum frequency softlimit is not revised as per the
efficient frequency value.

v2: Replaced the conditional expression operator with 'if' statement (Tom)
v3: Corrected the derivation of efficient frequency & shifted the
    GEN9_FREQ_SCALER multiplications downwards (Ville)

Issue: VIZ-5143
Signed-off-by: NAkash Goel <akash.goel@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Because the cool kids use dev_priv and FBC wants to be cool too.

We've been historically using struct drm_device on the FBC function
arguments, but we only really need it for intel_vgpu_active(): we can
use dev_priv everywhere else. So let's fully switch to dev_priv since
I'm getting tired of adding "struct drm_device *dev = dev_priv->dev"
everywhere.

If I get a NACK here I'll propose the opposite: convert all the
functions that currently take dev_priv to take dev.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Burning cpu cycles isn't awesome, so use sleeps instead.
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The hardware supposedly ignores the WM1 watermarks while the PND
deadline mode is enabled, but clear out the register just in case.
This is what the other OS does, and it does make register dumps look
more consistent when we don't have partial WM1 values lingering in
the registers (some WM1 watermarks already get zeroed when the actually
used DSPFW registers get written).
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NClint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Enabling PM5/DDR DVFS with multiple active pipes isn't a validated
configuration. It does seem to work most of the time at least, but
there is clearly an additional risk of underruns, so let's not play
with fire.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NClint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

CxSR (or maxfifo on VLV/CHV) blocks somne changes to the plane control
register (enable bit at least, not quite sure about the rest). So in
order to have the plane enable/disable when we want we need to first
kick the hardware out of cxsr.

Unfortunateloy this requires some extra vblank waits. For the CxSR
enable after the plane update we should eventually use an async
vblank worker, but since we don't have that just do sync vblank
waits. For the disable case we have no choice but to do it
synchronously.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NClint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In order to get decnet memory self refresh residency on VLV, flip it
over to the new CHV way of doing things. VLV doesn't do PM5 or DDR DVFS
so it's a bit simpler.

I'm not sure the currently memory latency used for CHV is really
appropriate for VLV. Some further testing will probably be needed to
figure that out.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NClint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Consider which planes are active and compute the FIFO split based on the
relative data rates. Since we only consider the pipe src width rather
than the plane width when computing watermarks it seems best to do the
same when computing the FIFO split as well. This means the only thing we
actually have to consider for the FIFO splut is the bpp, and we can
ignore the rest.

I've just stuffed the logic into the watermark code for now. Eventually
it'll need to move into the atomic update for the crtc.

There's also one extra complication I've not yet considered; Some of the
DSPARB registers contain bits related to multiple pipes. The registers
are double buffered but apparently they update on the vblank of any
active pipe. So doing the FIFO reconfiguration properly when multiple
pipes are active is not going to be fun. But let's ignore that mess for
now.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NClint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Turns out the VLV/CHV system agent doesn't understand memory
latencies, so trying to rely on the PND deadline mechanism is not
going to fly especially when DDR DVFS is enabled. Currently we try to
avoid the problems by lying to the system agent about the deadlines
and setting the FIFO watermarks to 8 cachelines. This however leads to
bad memory self refresh residency.

So in order to satosfy everyone we'll just give up on the deadline
scheme and program the watermarks old school based on the worst case
memory latency.

I've modelled this a bit on the ILK+ approach where we compute multiple
sets of watermarks for each pipe (PM2,PM5,DDR DVFS) and when merge thet
appropriate one later with the watermarks from other pipes. There isn't
too much to merge actually since each pipe has a totally independent
FIFO (well apart from the mess with the partially shared DSPARB
registers), but still decopuling the pipes from each other seems like a
good idea.

Eventually we'll want to perform the watermark update in two phases
around the plane update to avoid underruns due to the single buffered
watermark registers. But that's still in limbo for ILK+ too, so I've not
gone that far yet for VLV/CHV either.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NClint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Read out the current watermark settings from the hardware at driver init
time. This will allow us to compare the newly calculated values against
the currrent ones and potentially avoid needless WM updates.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NClint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We want cxsr exit to happen ASAP, so toss in some POSTING_READ()s to
make sure things are really kicked off.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NClint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Broxton is using a different register and different bit ordering
for rps status capabilities.

Also GT perf freqency register is different for Broxton so update
that.
Signed-off-by: NBob Paauwe <bob.j.paauwe@intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Leftover from the big purge

commit a5611654
Author: John Harrison <John.C.Harrison@Intel.com>
Date:   Thu Mar 5 14:03:03 2015 +0000

    drm/i915: Remove ironlake rc6 support

Cc: John Harrison <John.C.Harrison@Intel.com>
Cc: Rob Clark <robdclark@gmail.com>
Reported-by: NRob Clark <robdclark@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>

Bspec says we shouldn't enable IPS on BDW when the pipe pixel rate
exceeds 95% of the core display clock. Apparently this can cause
underruns.

There's no similar restriction listed for HSW, so leave that one alone
for now.

v2: Add pipe_config_supports_ips() (Chris)
v3: Compare against the max cdclk insted of the current cdclk
v4: Rebased to the latest
v5: Rebased to the latest
v6: Fix for patch style problems

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=83497Tested-by: NTimo Aaltonen <tjaalton@ubuntu.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMika Kahola <mika.kahola@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Rather than reading out the current cdclk value use the cached value we
have tucked away in dev_priv.

v2: Rebased to the latest
v3: Rebased to the latest
v4: Fix for patch style problems
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMika Kahola <mika.kahola@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Rename dpio_lock to sb_lock to inform the reader that its primary
purpose is to protect the sideband mailbox rather than some DPIO
state.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The primary plane frobbing was removed from the sprite code in
 commit ecce87ea
 Author: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
 Date:   Tue Apr 21 17:12:50 2015 +0300

    drm/i915: Remove implicitly disabling primary plane for now

but the intel_flush_primary_plane() calls were left behind. Replace them
with straight forward POSTING_READ() of the sprite surface address
register.

The other user of intel_flush_primary_plane() is g4x_disable_trickle_feed()
where we can just inline the steps directly.

This allows intel_flush_primary_plane() to be killed off.

Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In commit 1854d5ca
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date:   Tue Apr 7 16:20:32 2015 +0100

    drm/i915: Deminish contribution of wait-boosting from clients

we removed an atomic timer based check for allowing waitboosting and
moved it below the mutex taken during RPS. However, that mutex can be
held for long periods of time on Vallyview/Cherryview as communication
with the PCU is slow. As clients may frequently wait for results (e.g.
such as tranform feedback) we introduced contention between the client
and the RPS worker. We can take advantage of the RPS worker, by
switching the wait boost decision to use spin locks and defer the
actual reclocking to the worker.

Fixes a regression of up to 45% on Baytrail and Baswell!

v2 (Daniel):
- Use max_freq_softlimit instead of the not-yet-merged boost
  frequency.
- Don't inject a fake irq into the boost work, instead treat
  client_boost as just another legit waker.

v3: Drop the now unused mask (Chris).

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=90112
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> (v1)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

As Daniel commented on

commit b7ffe1362c5f468b853223acc9268804aa92afc8
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date:   Mon Apr 27 13:41:24 2015 +0100

    drm/i915: Free RPS boosts for all laggards

it is better to be explicit when sharing hardcoded values such as
throttle/boost timeouts. Make it so!
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

GTT caching was disabled by default on gen8 due to not working with
big pages. Some information suggests that it got fixed, but still
GTT caching has been left disabled by default. Or could be it just
meant that the default was changed to off, and hence the problem
got solved.

Enable GTT caching in the hopes of some performance increase.
Whether or not the big pages issue has been fixed is irrelevant
at this stage since we don't use big pages.

This gives me a 1-2% improvement in xonotic on my BSW. Haven't tried
BDW, but supposedly it has larger TLBs so might not benefit as much.
On HSW GTT caching is enabled by default.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

GEN8_L3SQCREG1 isn't saved in the context (verified by going through
a context dump), and so we shouldn't be using the ring w/a code to
initialize it. Also Bspec explicitly talks about MMIO and writing it
with the CPU.

Additionally there's another w/a WaTempDisableDOPClkGating:bdw which
tells us to disable DOP clock gating around the GEN8_L3SQCREG1 write
to make sure everyone notices the change. So let's do that as well.

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We're not using ilk_init_lp_watermarks() on BDW for some reason.
Probably due to the BDW patches and the relevant WM patches landing
roughlly at the same time. Fix it up.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

If the client stalls on a congested request, chosen to be 20ms old to
match throttling, allow the client a free RPS boost.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: s/rq/req/]
[danvet: s/0/NULL/ reported by 0-day build]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Now that we have internal clients, rather than faking a whole
drm_i915_file_private just for tracking RPS boosts, create a new struct
intel_rps_client and pass it along when waiting.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: s/rq/req/]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Since we will often pageflip to an active surface, we will often have to
wait for the surface to be written before issuing the flip. Also we are
likely to wait on that surface in plenty of time before the vblank.
Since we have a mechanism for boosting when a flip misses the expected
vblank, curtain the number of times we RPS boost when simply waiting for
mmioflip.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: s/rq/req/]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Ring switches can occur many times per frame, and are often out of
control, causing frequent RPS boosting for no practical benefit. Treat
the sw semaphore synchronisation as a separate client and only allow it
to boost once per busy/idle cycle.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: s/rq/req/]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The merged seqno->request conversion from John called request
variables req, but some (not all) of Chris' recent patches changed
those to just rq. We've had a lenghty (and inconclusive) discussion on
irc which is the more meaningful name with maybe at most a slight bias
towards req.

Given that the "don't change names without good reason to avoid
conflicts" rule applies, so lets go back to a req everywhere for
consistency. I'll sed any patches for which this will cause conflicts
before applying.

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: John Harrison <John.C.Harrison@Intel.com>
[danvet: s/origina/merged/ as pointed out by Chris - the first
mass-conversion patch was from Chris, the merged one from John.]
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>