Now that we have drm_planes for the cursor and primary we can move the
pixel_size handling into vlv_compute_drain_latency() and just pass the
appropriate plane to it.

v2: Check plane->state->fb instead of plane->fb

Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org> (v1)
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
[danvet: Resolve conflict with Matt's s/plane->fb/plane->state->fb/
patch.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Introduce struct vlv_wm_values to house VLV watermark/drain latency
values. We start by using it when computing the drain latency values.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Move the DDL precision handling into vlv_compute_drain_latency() so the
callers don't have to duplicate the same code to deal with it.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The current drain lantency computation relies on hardcoded limits to
determine when the to use the low vs. high precision multiplier.
Rewrite the code to use a more straightforward approach.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Kill the silly DRAIN_LATENCY_PRECISION_* defines and just use the raw
number instead.

v2: Move the sprite 32/16 -> 16/8 preision multiplier
    change to another patch (Jesse)
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Apparently we must yet halve the DDL drain latency from what we're
using currently. This little nugget is not in any spec, but came
down through the grapevine.

This makes the displays a bit more stable. Not quite fully stable but at
least they don't fall over immediately on driver load.

v2: Update high_precision in valleyview_update_sprite_wm() too (Jesse)
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Apparently, this has never worked reliably and is currently disabled. Also, the
gains are not particularly impressive. Thus rather than try to keep unused code
from decaying and having to update it for other driver changes, it was decided
to simply remove it.

For: VIZ-5115
Signed-off-by: NJohn Harrison <John.C.Harrison@Intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Implicit usage of local variables in macros isn't exactly the greatest
thing in the world, especially when that variable is the drm device and
we want to move towards a broader use of the i915 device structure.

Let's make for_each_plane() take dev_priv as its first argument then.
Suggested-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: Chris Wilson <chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

plane->fb is a legacy pointer that not always be up-to-date (or updated
early enough).  Make sure the watermark code uses plane->state->fb so
that we're always doing our calculations based on the correct
framebuffers.

This patch was generated by Coccinelle with the following semantic
patch:

        @@
        struct drm_plane *P;
        @@
        - P->fb
        + P->state->fb

v2: Rebase
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The cursor size fields in intel_crtc just duplicate the data from
cursor->state.crtc_{w,h} so we don't need them any more.  Worse, their
use in the watermark code actually introduces a subtle bug since they
don't get updated to mirror the state values until the plane commit
stage, which is *after* we've already used them to calculate new
watermark values.  This happens because we had to move watermark updates
slightly earlier (outside vblank evasion) in commit

        commit 32b7eeec
        Author: Matt Roper <matthew.d.roper@intel.com>
        Date:   Wed Dec 24 07:59:06 2014 -0800

            drm/i915: Refactor work that can sleep out of commit (v7)

Dropping the intel_crtc fields and just using the state values (which
are properly updated by the time watermark updates happen) should solve
the problem.

Aside from the actual removal of the struct fields (which are formatted
in a way that I couldn't figure out how to match in Coccinelle), the
rest of this patch was generated via the following semantic patch:

        // Drop assignment
        @@
        struct intel_crtc *C;
        struct drm_plane_state S;
        @@
        (
        - C->cursor_width = S.crtc_w;
        |
        - C->cursor_height = S.crtc_h;
        )

        // Replace usage
        @@
        struct intel_crtc *C;
        expression E;
        @@
        (
        - C->cursor_width
        + C->base.cursor->state->crtc_w
        |
        - C->cursor_height
        + C->base.cursor->state->crtc_h
        |
        - to_intel_crtc(E)->cursor_width
        + E->cursor->state->crtc_w
        |
        - to_intel_crtc(E)->cursor_height
        + E->cursor->state->crtc_h
        )

v2: Rebase

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Joe Konno <joe.konno@linux.intel.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=89346Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Display watermarks need different programming for different tiling
modes.

Set the relevant flag so this happens during the plane commit and
add relevant data into a structure made available to the watermark
computation code.

v2: Pass in tiling info to sprite plane updates as well.
v3: Rebased for plane handling changes.
v4: Handle fb == NULL when plane is disabled.
v5: Refactored for addfb2 interface.
v6: Refactored for fb modifier changes.
v7: Updated for atomic commit by only updating watermarks when tiling changes.
v8: BSpec watermark calculation updates.
v9: Restrict scope of y_tile_minimum variable. (Damien Lespiau)
v10: Get fb from plane state otherwise we are working on old state.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Acked-by: NAnder Conselvan de Oliveira <ander.conselvan.de.oliveira@intel.com>
Acked-by: NMatt Roper <matthew.d.roper@intel.com>
Reviewed-by: Damien Lespiau <damien.lespiau@intel.com> (v9)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Recent BSpect updates have changed the watermark calculation to avoid
display flickering in some cases.

v2: Fix check against DDB allocation and tidy the code a bit. (Damien Lespiau)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

I overlooked the fact that we need to allocate a minimum 8 blocks and
that just allocating the planes depending on how much they need to fetch
from the DDB in proportion of how much memory bw is necessary for the
whole display can lead to cases where we don't respect those minima (and
thus overrun).

So, instead, start by allocating 8 blocks to each active display plane
and then allocate the remaining blocks like before.

v2: Rebase on top of -nightly

Cc: Mahesh Kumar <mahesh1.kumar@intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This W/A is put in a gen9 specific function because it may well be
needed on other gen9 platforms.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Let's also take the opportunity the remove the comment telling it's a
pre-prod W/A, it should be obvious from the stepping test.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We'll gather cross-gen9 W/A in a separate function later.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

With Intel GVT-g, GPU power management is controlled by
host driver, so there is no need to provide virtualized
GPU PM support. In the future it might be useful to gather
VM input for freq boost, but now let's disable it simply.

v2:
take Chris' comments:
        - do not special case this to gen6+
Signed-off-by: NYu Zhang <yu.c.zhang@linux.intel.com>
Signed-off-by: NJike Song <jike.song@intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

When enabling new platforms, we may not have any W/A to apply,
especially that, now, a bunch of them have to be done from the ring.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Move Wa4x4STCOptimizationDisable to gen9_init_workarounds

v2: rebase
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2: Add stepping check for WaDisableSDEUnitClockGating.
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
[danvet: Rebase.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Move WaDisableDgMirrorFixInHalfSliceChicken5 to gen9_init_workarounds

v2: Added stepping check

v3: Removed unused register bitmap
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
[danvet: Bikesheds.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Replace the valleyview_set_rps() and gen6_set_rps() calls with
intel_set_rps() which itself does the IS_VALLEYVIEW() check. The
code becomes simpler since the callers don't have to do this check
themselves.

Most of the change was performe with the following semantic patch:
@@
expression E1, E2, E3;
@@
- if (IS_VALLEYVIEW(E1)) {
-  valleyview_set_rps(E2, E3);
- } else {
-  gen6_set_rps(E2, E3);
- }
+ intel_set_rps(E2, E3);

Adding intel_set_rps() and making valleyview_set_rps() and gen6_set_rps()
static was done manually. Also valleyview_set_rps() had to be moved a
bit avoid a forward declaration.

v2: Use a less greedy semantic patch

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Suggested-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The efficient frequency (RPe) should stay in the range
RPn <= RPe <= RP0.  The pcode clamps the returned value
internally on Broadwell but not on Haswell.

Fix for missing range check in
commit 93ee2920
Author: Tom O'Rourke <Tom.O'Rourke@intel.com>
Date:   Wed Nov 19 14:21:52 2014 -0800

    drm/i915: Use efficient frequency for HSW/BDW

Reference: http://lists.freedesktop.org/archives/intel-gfx/2015-February/059802.htmlReported-by: NMichael Auchter <a@phire.org>
Suggested-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: stable@vger.kernel.org # v3.19
Signed-off-by: NTom O'Rourke <Tom.O'Rourke@intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Move the CHV check into vlv_set_rps_idle() to simplify the caller a bit.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Replace all the vlv_gpu_freq(), vlv_freq_opcode(),
*GT_FREQUENCY_MULTIPLIER, and /GT_FREQUENCY_MULTIPLIER instances
with intel_gpu_freq() and intel_freq_opcode() calls.

Most of the change was performed with the following semantic patch:
@@
expression E;
@@
(
- E * GT_FREQUENCY_MULTIPLIER
+ intel_gpu_freq(dev_priv, E)
|
- E *= GT_FREQUENCY_MULTIPLIER
+ E = intel_gpu_freq(dev_priv, E)
|
- E /= GT_FREQUENCY_MULTIPLIER
+ E = intel_freq_opcode(dev_priv, E)
|
- do_div(E, GT_FREQUENCY_MULTIPLIER)
+ E = intel_freq_opcode(dev_priv, E)
)

@@
expression E1, E2;
@@
(
- vlv_gpu_freq(E1, E2)
+ intel_gpu_freq(E1, E2)
|
- vlv_freq_opcode(E1, E2)
+ intel_freq_opcode(E1, E2)
)

@@
expression E1, E2, E3, E4;
@@
(
- if (IS_VALLEYVIEW(E1)) {
-  E2 = intel_gpu_freq(E3, E4);
- } else {
-  E2 = intel_gpu_freq(E3, E4);
- }
+ E2 = intel_gpu_freq(E3, E4);
|
- if (IS_VALLEYVIEW(E1)) {
-  E2 = intel_freq_opcode(E3, E4);
- } else {
-  E2 = intel_freq_opcode(E3, E4);
- }
+ E2 = intel_freq_opcode(E3, E4);
)

One hunk was manually undone as intel_gpu_freq() ended up
calling itself. Supposedly it would be possible to exclude
certain functions via !=~, but I couldn't get that to work.

Also the removal of vlv_gpu_freq() and vlv_opcode_freq() compat
wrappers was done manually.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Rename the vlv_gpu_freq() and vlv_freq_opecode() functions to have
an intel_ prefix, and handle non-VLV/CHV platforms in them as well.
Leave the vlv_ names around for now since they're currently used.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We set the WIZ hashing mode to 16x4 for all the other gen6+
platfotrms, so let's follow suit on VLV.

My VLV is AWOL currently so I didn't test this, but since the results
for all the other platforms agree that 16x4 is the fastest we might
assume the same holds for VLV.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NArun Siluvery <arun.siluvery@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Drop WaDisablePwrmtrEvent:chv as it's no longer needed.

Also remove the WaSetMaskForGfxBusyness:chv note, but we still
leave the GEN6_RP_MEDIA_IS_GFX bit enabled as that's still the
recommended setting.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NArun Siluvery <arun.siluvery@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Enable coarse power gating for Gen9. This feature allows render and
media engine to enter RC6 independently. Policies are configured
together with RC6. This feature will only be enabled when RC6 is
enabled.

v2: Rebase after Chris'/Mika's forcewake change (Damien)
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NZhe Wang <zhe1.wang@intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2: Use the new function, gen6_init_rps_frequencies() (Damien)

Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com> (v1)
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Per latest PM programming guide.

v2: the wrong flavour of the function updating the ring frequency was
    called, leading to dead locks (Tvrtko)

v3: Add GEN6_RP_MEDIA_IS_GFX to RP_CONTROL (Imre, done by Damien)
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
[danvet: Fixup conflicts with Mika's forcewake refactor.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We have multiple forcewake domains now on recent gens. Change the
function naming to reflect this.

v2: More verbose names (Chris)
v3: Rebase
v4: Rebase
v5: Add documentation for forcewake_get/put
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: Deepak S <deepak.s@linux.intel.com> (v2)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

To match the semantics of drm_crtc->state, which this will eventually
become. The allocation of the memory for config will be fixed in a
followup patch. By adding the extra _config field to intel_crtc it was
possible to generate this entire patch with the cocci script below.

@@ @@
struct intel_crtc {
...
-struct intel_crtc_state config;
+struct intel_crtc_state _config;
+struct intel_crtc_state *config;
...
}
@@ struct intel_crtc *crtc; @@
-memset(&crtc->config, 0, sizeof(crtc->config));
+memset(crtc->config, 0, sizeof(*crtc->config));
@@ @@
__intel_set_mode(...) {
<...
-to_intel_crtc(crtc)->config = *pipe_config;
+(*(to_intel_crtc(crtc)->config)) = *pipe_config;
...>
}
@@ @@
intel_crtc_init(...) {
...
WARN_ON(drm_crtc_index(&intel_crtc->base) != intel_crtc->pipe);
+intel_crtc->config = &intel_crtc->_config;
return;
...
}
@@ struct intel_crtc *crtc; @@
-&crtc->config
+crtc->config
@@ struct intel_crtc *crtc; identifier member; @@
-crtc->config.member
+crtc->config->member
@@ expression E; @@
-&(to_intel_crtc(E)->config)
+to_intel_crtc(E)->config
@@ expression E; identifier member; @@
-to_intel_crtc(E)->config.member
+to_intel_crtc(E)->config->member

v2: Clarify manual changes by splitting them into another patch. (Matt)
    Improve cocci script to generate even more of the changes. (Ander)
Signed-off-by: NAnder Conselvan de Oliveira <ander.conselvan.de.oliveira@intel.com>
Reviewed-by: NMatt Roper <matthew.d.roper@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

And get rid of the duplicate mode structures. This patch was generated
with the following semantic patch:

@@ @@
struct intel_crtc_state {
+struct drm_crtc_state base;
+
...
-struct drm_display_mode requested_mode;
-struct drm_display_mode adjusted_mode;
...
}
@@ struct intel_crtc_state *state; @@
-state->adjusted_mode
+state->base.adjusted_mode
@@ struct intel_crtc_state *state; @@
-state->requested_mode
+state->base.mode
@@ struct intel_crtc_state state; @@
-state.adjusted_mode
+state.base.adjusted_mode
@@ struct intel_crtc_state state; @@
-state.requested_mode
+state.base.mode
@@ struct drm_crtc *crtc; @@
-to_intel_crtc(crtc)->config.adjusted_mode
+to_intel_crtc(crtc)->config.base.adjusted_mode
@@ identifier member; expression E; @@
-PIPE_CONF_CHECK_FLAGS(adjusted_mode.member, E);
+PIPE_CONF_CHECK_FLAGS(base.adjusted_mode.member, E);
@@ identifier member; @@
-PIPE_CONF_CHECK_I(adjusted_mode.member);
+PIPE_CONF_CHECK_I(base.adjusted_mode.member);
@@ identifier member; @@
-PIPE_CONF_CHECK_CLOCK_FUZZY(adjusted_mode.member);
+PIPE_CONF_CHECK_CLOCK_FUZZY(base.adjusted_mode.member);

v2: Completely generate the patch with cocci. (Ander)
Signed-off-by: NAnder Conselvan de Oliveira <ander.conselvan.de.oliveira@intel.com>
Reviewed-by: NMatt Roper <matthew.d.roper@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The objective is to make this structure usable with the atomic helpers,
so let's start with the rename. Patch generated with coccinelle:

@@ @@
-struct intel_crtc_config {
+struct intel_crtc_state {
...
}
@@ @@
-struct intel_crtc_config
+struct intel_crtc_state

v2: Completely generate the patch with cocci. (Ander)
Signed-off-by: NAnder Conselvan de Oliveira <ander.conselvan.de.oliveira@intel.com>
Reviewed-by: NMatt Roper <matthew.d.roper@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The performance regression from the CHV RC6 EI->TO change is now fixed
so re-enable TO mode for better RC6 resicency.

This reverts commit e85a5c79.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Deepak S<deepak.s@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>