CHV wants even rps opcodes so print a warning of the
min/max/rpe/rp1 values are odd, and warn if an odd value
slips through to valleyview_set_rps() and truncate it to
an even value.

Also add a comment to chv_freq_opcode() to make sure no one
changes the code without considering this requirement.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDeepak S <deepak.s@linux.intel.com>
[danvet: Help git along in applying the patch, somehow it silently
ended up in the vlv init_gt_powersave function.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

BDW supports GT C0 residency reporting in constant time unit. Driver
calculates GT utilization based on C0 residency and adjusts RP
frequency up/down accordingly. For offscreen workload specificly,
set frequency to RP0.

Offscreen task is not restricted by frame rate, it can be
executed as soon as possible. Transcoding and serilized workload
between CPU and GPU both need high GT performance, RP0 is a good
option in this case. RC6 will kick in to compensate power
consumption when GT is not active.

v2: Rebase on recent drm-intel-nightly
v3: Add flip timerout monitor, when no flip is deteced within
100ms, set frequency to RP0.
Signed-off-by: NDaisy Sun <daisy.sun@intel.com>
[torourke: rebased on latest and resolved conflict]
Signed-off-by: NTom O'Rourke <Tom.O'Rourke@intel.com>
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

init_clock_gating() is too late to read out the mem_freq. We already
want to print out the GPU MHz numbers before it's called. Move the
mem_freq setup to init_gt_powersave().

v2: Also kill the CHV_CZ_CLOCK_FREQ_MODE_* defines
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDeepak S <deepak.s@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

For BDW workarounds are currently initialized in init_clock_gating() but
they are lost during reset, suspend/resume etc; this patch moves the WAs
that are part of register state context to render ring init fn otherwise
default context ends up with incorrect values as they don't get initialized
until init_clock_gating fn.

v2: Add workarounds to golden render state
This method has its own issues, first of all this is different for
each gen and it is generated using a tool so adding new workaround
and mainitaining them across gens is not a straightforward process.

v3: Use LRIs to emit these workarounds (Ville)
Instead of modifying the golden render state the same LRIs are
emitted from within the driver.

v4: Use abstract name when exporting gen specific routines (Chris)

For: VIZ-4092
Signed-off-by: NArun Siluvery <arun.siluvery@linux.intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

According to spec FBC on BDW and HSW are identical without any gaps.
So let's copy the nuke and let FBC really start compressing stuff.

Without this patch we can verify with false color that nothing is being
compressed. With the nuke in place and false color it is possible
to see false color debugs.

Unfortunatelly on some rings like BCS on BDW we have to avoid Bits 22:18 on
LRIs due to a high risk of hung. So, when using Blt ring for frontbuffer rend
cache would never been cleaned and FBC would stop compressing buffer.
One alternative is to cache clean on software frontbuffer tracking.

v2: Fix rebase conflict.
v3: Do not clean cache on BCS ring. Instead use sw frontbuffer tracking.
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Because CHV uses cherryview_init_clock_gating instead of
gen8_init_clock_gating.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Because BDW has WPT, which is equivalent to LPT. This is just like the
CPT/PPT case.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We need do forcewake before Disabling RC6, This is what the BIOS
expects while going into suspend.

v2: updated commit message. (Daniel)

Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDeepak S <deepak.s@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Chris has decided that enough is enough. It's time to fixup dev Vs
dev_priv. This is a modest contribution to the crusade.

v2: Still use INTEL_INFO(), for the (mythical!) case we want to hardcode
    the info struct with defines (Chris)
    Rename the macro argument from 'dev' to 'dev_priv' (Jani)

v3: Use names unlikely to be used as macro arguments (Chris)
Suggested-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Primary planes support 180 degree rotation. Expose the feature
through rotation drm property.

v2: Calculating linear/tiled offsets based on pipe source width and
height. Added 180 degree rotation support in ironlake_update_plane.

v3: Checking if CRTC is active before issueing update_plane. Added
wait for vblank to make sure we dont overtake page flips. Disabling
FBC since it does not work with rotated planes.

v4: Updated rotation checks for pending flips, fbc disable. Creating
rotation property only for Gen4 onwards. Property resetting as part
of lastclose.

v5: Resetting property in i915_driver_lastclose properly for planes
and crtcs. Fixed linear offset calculation that was off by 1 w.r.t
width in i9xx_update_plane and ironlake_update_plane. Removed tab
based indentation and unnecessary braces in intel_crtc_set_property
and intel_update_fbc. FBC and flip related checks should be done only
for valid crtcs.

v6: Minor nits in FBC disable checks for comments in intel_crtc_set_property
and positioning the disable code in intel_update_fbc.

v7: In case rotation property on inactive crtc is updated, we return
successfully printing debug log as crtc is inactive and only property change
is preserved.

v8: update_plane is changed to update_primary_plane, crtc->fb is changed to
crtc->primary->fb  and return value of update_primary_plane is ignored.

v9: added rotation property to primary plane instead of crtc. Removing reset
of rotation property from lastclose. rotation_property is moved to
drm_mode_config, so drm layer will take care of resetting. Adding updation of
fbc when rotation is set to 0. Allowing rotation only if value is
different than old one.

v10: Calling intel_primary_plane_setplane instead of update_primary_plane in
set_property(Daniel).

v11: Using same set_property function for both primary and sprite, Adding
primary plane specific code in the same function (Matt).

v12: Removing disabling/ enabling of fbc from set_property because it is done
from intel_pipe_set_base. Other formatting

v13: we need to call disable_fbc before changing the rotation to 180,
disable_fbc from intel_pipe_set_base gets called very late, that will
be used to re-enable fbc if rotation is set to 0 (Ville).

Testcase: igt/kms_rotation_crc
Signed-off-by: NUma Shankar <uma.shankar@intel.com>
Signed-off-by: NSagar Kamble <sagar.a.kamble@intel.com>
Signed-off-by: NSonika Jindal <sonika.jindal@intel.com>
[danvet: Add FIXME to explain why we need the open-coded update_fbc
hunk to disable fbc when rotated 180 degree. And make checkpatch
happier.]
Acked-by: NMatt Roper <matthew.d.roper@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Program DDL register as part of sprite watermark programming for CHV and VLV.

v2: Rename DRAIN_LATENCY_MAX by DRAIN_LATENCY_MASK

v3: Addressed review comments by Ville
    - Changed Sprite DDL definitions to more generic to avoid multiple if-else
    - Changed bit masking to customary form
    - Changed to bitwise shorthand operator for sprite_dl assignment
Signed-off-by: NGajanan Bhat <gajanan.bhat@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Round up clock computation and limit drain latency to maximum of 0x7F.
Signed-off-by: NGajanan Bhat <gajanan.bhat@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Modify drain latency computation to use it for any plane. Same function can be
used for primary, cursor and sprite planes.

v2: Adressed review comments by Imre and Ville.
    - Moved clock round up in separate patch
    - Added WARN check for clock and pixel size
    - Simplified bit masking
    - Use cursor_base instead of reg read

v3: Changed to bitwise shorthand operator for plane_dl assignment.
Signed-off-by: NGajanan Bhat <gajanan.bhat@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Replace the semi-funky cmnlane assert/deassert macros with something a
bit more conventional. Also protect the macro arguments properly (also
for  PHY_POWERGOOD()).
Reviewed-by: NRafael Barbalho <rafael.barbalho@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It looks like frobbing the cmnreset line on pne PHY disturbs the other
PHY on chv. The result is a black screen. On HDMI it's just a flash of
black, but DP usually falls over and can't get back up.

As a workaround set up the power domains so that both common lane
wells power up and down together. I also tried leaving the cmnreset
deasserted even the if the power well goes down but that didn't seem
acceptable to the PHY.
Reviewed-by: NRafael Barbalho <rafael.barbalho@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

CHV has a third pipe so we need to compute the watermarks for its
planes. Add cherryview_update_wm() to do just that.

v2: Rebase on top of Imre's cxsr changes
v3: Pass crtc to vlv_update_drain_latency()
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Instead of looping through all CRTCs, update DDL for current CRTC for which
watermark is being updated.
CHV is confirmed to have precision of 32/64 which is same as VLV.
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NGajanan Bhat <gajanan.bhat@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The VLV/CHV DDL registers are uniform, and neatly enough the register
offsets are sane so we can easily unify them to a single set of defines
and just pass the pipe as the parameter to compute the register offset.

Note that we now fill out the drain latency for pipe C on CHV which we
didn't do before. The rest of the pipe C watermarks are still untouched
but that will be remedied later by adding a proper cherryview_update_wm()
function.

v2: Add a note about CHV pipe C changes (Paulo)
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Add defines for all the watermark registers on modernish gmch platforms.

VLV has increased the number of bits available for certain watermaks so
expand the masks appropriately. Also vlv and chv have added some extra
FW registers.

Not sure what happened on chv because a new register called FW9 is now
at the offset where FW7 was on vlv, while FW7 and FW8 (another new
register) have been moved off somewhere else. Oh well, well just need
two defines for FW7 then.

v2: Fix DSPHOWM1 offset (Paulo)
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

With this bit enabled, HW changes the color when compressing frames for
debug purposes.

ALthough the simple way to enable a single bit is over intel_reg_write,
this value is overwriten on next update_fbc so depending on the workload
it is not possible to set this bit with intel-gpu-tools. So this patch
introduces a persistent way to enable false color over debugfs.

v2: Use DEFINE_SIMPLE_ATTRIBUTE as Daniel suggested
v3: (Ville) only do false color for IVB+ since according to spec bit is
    MBZ before IVB.
v4: We don't have FBC on valleyview nor on cherryview (Ben)
v5: s/!HAS_PCH_SPLIT/!HAS_FBC (Ville)

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Split some WM debug prints to multiple lines. This shouldn't hurt
grappability since the important part is at the start and the rest
is just repeated stuff for each pipe.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Add the TX wells for port D. The Punit subsystem numbers are a total
guess at this time. Also I'm not sure these even exist. Certainly the
Punit in current hardware doesn't deal with these.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Add the TX wells for ports B and C just like on VLV.

Again Punit doesn't seem ready (or the wells don't even exist anymore)
so leave it iffed out.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

CHV has a power well for each pipe. Add the code to deal with them.

The Punit in current hardware doesn't seem ready for this yet, so
leave it iffed out.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Not sure if it's still there since chv has per-pipe power wells.
At least with current Punit this doesn't work. Also the display
irq handling would need to be adjusted for pipe C. So leave the
code iffed out for now.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

CHV has two display PHYs so there are also two cmnlane power wells. Add
the approriate code to power the wells up/down.

Like on VLV we do the cmnreset assert/deassert and the DPLL refclock
enabling at approriate times.

This code actually works on my bsw.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Add chv_power_wells[] so we can start to build up the power well support
for chv. Just the "always on" well there initialy.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Tested-by: NRafael Barbalho <rafael.barbalho@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The DDL registers can hold 7bit numbers. Make the most of those seven
bits by adjusting the threshold where we switch between the 64 vs. 32
precision multipliers.

Also we compute 'entries' to make the decision about precision, and then
we recompute the same value to calculate the actual drain latency. Just
use the already calculate 'entries' there.
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NZhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Users often can't do anything about this since their vendors stopped
providing BIOS updates. Also we seem to be able to hack around it
with increased latency values, and thus far the only reports have
been for screens with really high resolutions. So tune it down to a
level where only developers can see it.

Also drop some of the end-user fluff.
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Power users spot this and then get adventurous and try to adjust
module driver options. Nothing good ever came out of that, so
hide it better.
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Use ktime_get_raw_ns() and get rid of the back and forth timespec
conversions.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NJohn Stultz <john.stultz@linaro.org>

Now that we use the runtime IRQ enable/disable functions in our suspend
path, we can simply check the pm._irqs_disabled flag everywhere.  So
rename it to catch the users, and add an inline for it to make the
checks clear everywhere.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Move it from hsw_power_well_post_enable() (intel_pm.c) to i915_irq.c
so we can reuse the nice IRQ macros we have there. The main difference
is that now we're going to check if the IIR register is non-zero when
we try to re-enable the interrupts.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Traditionally we use genX_ for GT/render stuff and the codenames for
display stuff. But the gt and pm interrupt handling functions on
gen5/6+ stuck out as exceptions, so convert them.

Looking at the diff this nicely realigns our ducks since almost all
the callers are already platform-specific functions following the
genX_ pattern.

Spotted while reviewing some internal rps patches.

No function change in this patch.
Acked-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In the future, we'll need the height of the fb to fetch from memory for
WM computation.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Drop WaGsvBringDownFreq on CHV.
When in RC6 requesting the min freq should be fine to bring the
voltage down.
Signed-off-by: NDeepak S <deepak.s@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@Virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We might be leaving the GPU Frequency (and thus vnn) high during the suspend.
Force gt to move to lowest freq while suspending.

v2: Fixed typo in commit message (Deepak)

v3: Force gt to lowest freq in suspend_gt_powersave (Daniel)

v4: Add GPU min freq set _after_ we've cancelled the rps works (Daniel)
Signed-off-by: NDeepak S <deepak.s@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

CC: Deepak S <deepak.s@linux.intel.com>
CC: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NFengguang Wu <fengguang.wu@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>