Let's make all the "do we have enough DDB for this WM level?"
checks use min_ddb_alloc. To achieve that we need to populate
this for the transition watermarks as well.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210305153610.12177-5-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

For non-transition watermarks we are supposed to check min_ddb_alloc
rather than plane_res_b when determining if we have enough DDB space
for it. A bit too much copy pasta made me check the wrong thing.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Fixes: df4a50a3 ("drm/i915: Zero out SAGV wm when we don't have enough DDB for it")
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210305153610.12177-4-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

Say we have two planes enabled with watermarks configured
as follows:
plane A: wm0=enabled/can_sagv=false, wm1=enabled/can_sagv=true
plane B: wm0=enabled/can_sagv=true,  wm1=disabled

This is possible since the latency we use to calculate
can_sagv may not be the same for both planes due to
skl_needs_memory_bw_wa().

In this case skl_crtc_can_enable_sagv() will see that
both planes have enabled at least one watermark level
with can_sagv==true, and thus proceeds to allow SAGV.
However, since plane B does not have wm1 enabled
plane A can't actually use it either. Thus we are
now running with SAGV enabled, but plane A can't
actually tolerate the extra latency it imposes.

To remedy this only allow SAGV on if the highest common
enabled watermark level for all active planes can tolerate
the extra SAGV latency.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210305153610.12177-3-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

On HSW/BDW with VT-d active the first tile row scanned out
after the first async flip of the frame often ends up corrupted.

Whether the corruption happens or not depends on the scanline
on which the async flip happens, but the behaviour seems very
consistent. Ie. the same set of scanlines (which are most scanlines)
always show the corruption. And another set of scanlines (far less
of them) never shows the corruption.

I discovered that disabling the fetch-stride stretching
feature cures the corruption. This is some kind of TLB related
prefetch thing AFAIK. We already disable it on SNB primary
planes due to a documented workaround. The hardware folks
indicated that disabling this should be fine, so let's go
with that.

And while we're here, let's document the relevant bits on all
pre-skl platforms.

Fixes: 2a636e24 ("drm/i915: Implement async flip for ivb/hsw")
Fixes: cda195f1 ("drm/i915: Implement async flips for bdw")
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210220103303.3448-1-ville.syrjala@linux.intel.comReviewed-by: NKarthik B S <karthik.b.s@intel.com>

We know which WM0 (normal vs. SAGV) we supposedly programmed
into the hardware, so just check against that instead of accepting
either watermark as valid.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210226153204.1270-7-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

Seems to me that if we calculate WM0 using the bumped up SAGV latency
we need to calculate the transition watermark accordingly. Track it
alongside the other watermarks.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210226153204.1270-6-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

We'll want a SAGV transition watermark as well. Prepare
for that by collecting SAGV wm0 into a sub-strcture.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210226153204.1270-5-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

Let's consider sagv_wm0 as well when deciding whether to dump
out the watermark changes.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210226153204.1270-4-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

Let's handle the SAGV WM0 more like the other wm levels and just
totally zero it out when we don't have the DDB space to back it
up.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210226153204.1270-3-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

When we switch between SAGV on vs. off we need to reprogram all
plane wateramrks accordingly. Currently skl_wm_add_affected_planes()
totally ignores the SAGV watermark and just assumes we will use
the normal WM0.

Fix this by utilizing skl_plane_wm_level() which picks the
correct watermark based on use_sagv_wm. Thus we will force
an update on all the planes whose watermark registers need
to be reprogrammed.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210226153204.1270-2-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

There is nothing else to be executed after this if block.
Signed-off-by: NJosé Roberto de Souza <jose.souza@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210212182201.155043-2-jose.souza@intel.com

Rework the plane init calls to do the gen test one level higher.

Rework some of the plane helpers so they can live in new file,
there is still some scope to clean up the plane/fb interactions
later.

v2: drop atomic code back, rename file to Ville suggestions,
add header file.
v3: move scaler bits back
v4: drop wrong new includes (Ville)
v5: integrate the ccs gen12 changes
v6: fix unrelated code movement (Ville)
Signed-off-by: NDave Airlie <airlied@redhat.com>
[Jani: fixed up sparse warnings.]
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Reported-by: Nkernel test robot <lkp@intel.com>
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/4e88a5c6b9ab3b93cc2b6c7d78c26ae86f6abbd0.1612536383.git.jani.nikula@intel.com

As it now it is always required for GEN12+ the is_16gb_dimm name
do not make sense for GEN12+.

v2:
- Updated comment on top of "dram_info->wm_lv_0_adjust_needed =
!IS_GEN9_LP(i915);"
Reviewed-by: NLucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: NJosé Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210128164312.91160-3-jose.souza@intel.com

Readout the dbuf related stuff during driver init/resume and
stick it into our dbuf state.

v2: Keep crtc_state->wm.skl.ddb
Reviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210122205633.18492-9-ville.syrjala@linux.intel.com

In order to make the dbuf state computation less fragile
let's make it stand on its own feet by not requiring someone
to peek into a crystall ball ahead of time to figure out
which pipes need to be added to the state under which potential
future conditions. Instead we compute each piece of the state
as we go along, and if any fallout occurs that affects more than
the current set of pipes we add the affected pipes to the state
naturally.

That requires that we track a few extra thigns in the global
dbuf state: dbuf slices for each pipe, and the weight each
pipe has when distributing the same set of slice(s) between
multiple pipes. Easy enough.

We do need to follow a somewhat careful sequence of computations
though as there are several steps involved in cooking up the dbuf
state. Thoguh we could avoid some of that by computing more things
on demand instead of relying on earlier step of the algorithm to
have filled it out. I think the end result is still reasonable
as the entire sequence is pretty much consolidated into a single
function instead of being spread around all over.

The rough sequence is this:
1. calculate active_pipes
2. calculate dbuf slices for every pipe
3. calculate total enabled slices
4. calculate new dbuf weights for any crtc in the state
5. calculate new ddb entry for every pipe based on the sets of
   slices and weights, and add any affected crtc to the state
6. calculate new plane ddb entries for all crtcs in the state,
   and add any affected plane to the state so that we'll perform
   the requisite hw reprogramming

And as a nice bonus we get to throw dev_priv->wm.distrust_bios_wm
out the window.

v2: Keep crtc_state->wm.skl.ddb
Reviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210122205633.18492-8-ville.syrjala@linux.intel.com

Extract the code to calculate the weights used to chunk up the dbuf
between pipes. There's still extra stuff in there that shouldn't be
there and must be moved out, but that requires a bit more state to
be tracked in the dbuf state.

v2: Keep crtc_state->wm.skl.ddb
Reviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210122205633.18492-7-ville.syrjala@linux.intel.com

The dbuf state will be where we collect all the inter-pipe dbuf
allocation stuff. Start by adding the actual per-pipe ddb entries
there.

Originally the plan was to move them there outright, but that no longer
works as we're no longer guaranteed to have a dbuf state when it comes
time to sanity check the ddb overlaps in skl_commit_modeset_enables().
I think when I wrote this originally we did the watermark/ddb
calculation last, and so we couldn't have any crtcs in the state w/o
also having the dbuf state. But that has since changed and we do the
watermark/ddb calculation much earlier, and thus it is now possible
to commit crtcs w/o a dbuf state. So we keep another copy of the
information in the crtc state.

v2: Rebase
v3: Duplicate the entries instead of moving
Reviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210122205633.18492-6-ville.syrjala@linux.intel.com

Generalize icl_get_first_dbuf_slice_offset() into something that
just gives us the start+end of the dbuf chunk covered by the
specified slices as a standard ddb entry. Initial idea was to use
it during readout as well, but we shall see.
Reviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210122205633.18492-5-ville.syrjala@linux.intel.com

Put the code into a function with a descriptive name. Also relocate
the code a bit help future work.
Reviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210122205633.18492-4-ville.syrjala@linux.intel.com

skl_compute_dbuf_slices() has no use for the crtc state, so
just pass the crtc itself.
Reviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210122205633.18492-3-ville.syrjala@linux.intel.com

skl_ddb_get_pipe_allocation_limits() doesn't care how the weights
for distributing the ddb are caclculated for each pipe. Put that
calculation into a separate function so that such mundane details
are hidden from view.

v2: s/adjusted_mode/pipe_mode/
Reviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210122205633.18492-2-ville.syrjala@linux.intel.com

TGL adds another level of indirection for applying WA based on stepping
information rather than PCI REVID. So change TGL_REVID enum into
stepping enum and use PCI REVID as index into revid to stepping table to
fetch correct display and GT stepping for application of WAs as
suggested by Matt Roper.

Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: NAditya Swarup <aditya.swarup@intel.com>
Reviewed-by: NMatt Roper <matthew.d.roper@intel.com>
Signed-off-by: NLucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210119192931.1116500-1-lucas.demarchi@intel.com

DG1 is missing those two WA so instead of copy and paste it to the DG1
function, here calling the function that implements it.

While at it also renaming tgl_init_clock_gating to
gen12lp_init_clock_gating as it is also used by DG1, RKL and ADL-S.

Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: NJosé Roberto de Souza <jose.souza@intel.com>
Reviewed-by: NMatt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210113133759.72055-1-jose.souza@intel.com

Arguably some of these should use intel_de_read() or intel_de_write(),
however not all. Prioritize I915_READ() and I915_WRITE() removal in
general over migrating to the pedantically correct replacements right
away.
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201130111601.2817-7-jani.nikula@intel.com

Some media power gates are disabled by default. commit 5d869230
("drm/i915/tgl: Enable VD HCP/MFX sub-pipe power gating")
tried to enable it, but it duplicated an existent register.
So, the main PG setup sequences ended up overwriting it.

So, let's now merge this to the main PG setup sequence.

v2: (Chris): s/BIT/REG_BIT, remove useless comment,
    	     remove useless =0, use the right gt,
	     remove rc6 sequence doubt from commit message.

Fixes: 5d869230 ("drm/i915/tgl: Enable VD HCP/MFX sub-pipe power gating")
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: stable@vger.kernel.org#v5.5+
Cc: Dale B Stimson <dale.b.stimson@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201111072859.1186070-1-rodrigo.vivi@intel.com
(cherry picked from commit 695dc55b)
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>

Replace skl_adjusted_plane_pixel_rate() with the generic
intel_plane_pixel_rate(). The two should produce identical
results.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201106173042.7534-7-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

Store the relative data rate for planes in the crtc state
so that we don't have to use
intel_atomic_crtc_state_for_each_plane_state() to compute
it even for the planes that are no part of the current state.

Should probably just nuke this stuff entirely an use the normal
plane data rate instead. The two are slightly different since this
relative data rate doesn't factor in the actual pixel clock, so
it's a bit odd thing to even call a "data rate". And since the
watermarks are computed based on the actual data rate anyway
I don't really see what the point of this relative data rate
is. But that's for the future...
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201106173042.7534-6-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

In order to remove intel_atomic_crtc_state_for_each_plane_state()
from skl_crtc_can_enable_sagv() we can simply precompute whether
each wm level can tolerate the SAGV block time latency or not.

This has the nice side benefit that we remove the duplicated
wm level latency calculation. In fact the copy of that code
we had in skl_crtc_can_enable_sagv() didn't even handle
WaIncreaseLatencyIPCEnabled/Display WA #1141 whereas the copy
in skl_compute_plane_wm() did. So now we just have the one
copy which handles all the w/as.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201106173042.7534-5-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

intel_atomic_crtc_state_for_each_plane_state() peeks at the
plane's current state without holding the plane's mutex, trusting
that the crtc's mutex will protect it. In practice that does work
since our planes can't move between pipes, but it sets a bad
example. intel_atomic_crtc_state_for_each_plane_state() also
relies on crtc_state.uapi.plane_mask which may be full of lies
when it comes to the bigjoiner stuff, so soon we can't use it as
is anyway. So best to just get rid of it entirely. Which we can
easily do by switching to the g4x/vlv "raw" watermark approach.

Later on we should even be able to move the "raw" watermark
computation into the normal .plane_check() code, leaving only
the merging/clamping of the final watermarks to the later
stages. But that will require adjusting the ilk+ wm code
similarly as well.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NManasi Navare <manasi.d.navare@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201106173042.7534-3-ville.syrjala@linux.intel.com

Pass the whole intel_atomic_state to skl_build_pipe_wm()
and skl_allocate_pipe_ddb() so we can start to iterate
stuff containerd in the commit.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NManasi Navare <manasi.d.navare@intel.com>
Signed-off-by: NManasi Navare <manasi.d.navare@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201106173042.7534-2-ville.syrjala@linux.intel.com

With bigjoiner, there will be 2 pipes driving 2 halves of 1 transcoder,
because of this, we need a pipe_mode for various calculations, including
for example watermarks, plane clipping, etc.

v10:
* remove redundant pipe_mode assignment (Ville)
v9:
* pipe_mode in state dump nd state check (Ville)
v8:
* Add pipe_mode in readout in verify_crtc_state (Ville)
v7:
* Remove redundant comment (Ville)
* Just keep mode instead of pipe_mode (Ville)
v6:
* renaming in separate function, only pipe_mode here (Ville)
* Add description (Maarten)
v5:
* Rebase (Manasi)
v4:
* Manual rebase (Manasi)
v3:
* Change state to crtc_state, fix rebase err  (Manasi)
v2:
* Manual Rebase (Manasi)
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: NManasi Navare <manasi.d.navare@intel.com>
Reviewed-by: NAnimesh Manna <animesh.manna@intel.com>
[vsyrjala:
* Fix state checker
* Fix state dump
* Use pipe_mode for linetime watermarks
* Make sure pipe_mode normal timings are correct since the
  silly ddb code uses them
* Drop the redundant pipe_mode copies from intel_modeset_pipe_config()
  and intel_crtc_copy_uapi_to_hw_state()
* Use drm_mode_copy() all over]
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201112191718.16683-7-ville.syrjala@linux.intel.com

Some media power gates are disabled by default. commit 5d869230
("drm/i915/tgl: Enable VD HCP/MFX sub-pipe power gating")
tried to enable it, but it duplicated an existent register.
So, the main PG setup sequences ended up overwriting it.

So, let's now merge this to the main PG setup sequence.

v2: (Chris): s/BIT/REG_BIT, remove useless comment,
    	     remove useless =0, use the right gt,
	     remove rc6 sequence doubt from commit message.

Fixes: 5d869230 ("drm/i915/tgl: Enable VD HCP/MFX sub-pipe power gating")
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: stable@vger.kernel.org#v5.5+
Cc: Dale B Stimson <dale.b.stimson@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: https://patchwork.freedesktop.org/patch/msgid/20201111072859.1186070-1-rodrigo.vivi@intel.com

Remove the hand rolled array of WM0_PIPE register offsets
and use the standard _MMIO_PIPE3() instead.

v2: Take care of gvt too
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20181212211738.27770-1-ville.syrjala@linux.intel.comReviewed-by: NLucas De Marchi <lucas.demarchi@intel.com>

WAC6entrylatency is trying to fix excessive rc6 entry latency caused
by the extra delay from FBC_LLC_READ_CTRL, which is there for some
extra sync with uncore for frame buffer caching in LLC.

Reading through the hsd the recommendation was to set the FBC_LLC_FULLY_OPEN
bit to disable this extra delay entirely. This can be done whenever fb LLC
caching is not used. The alternative suggestion was to reduce the delay to
eg. 0x5 via updated BIOS programming instructions. But all the kbl/cfl
machines I've seen still have the default 0xff programmed. As we never use
fb LLC caching let's just apply the w/a to all skl derivatives to get
consistent rc6 latencies.

I was able to measure the effect of FBC_LLC_READ_CTRL to rc6 latency
via forcewake. Here's a graph of some of the results:

             sleep;fw_req=1;wait fw_ack==1;sleep;fw_req=0;wait fw_ack==0
 fw_ack==1 duration
    160us +----------------------------------------------------------------+
          |          +          +        $$+         +          +          |
          |  $$           $    $   ******$$ **   $ $**$*  #########$$######|
    140us |-$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$*$$$$$$$$$$$$$$$$ $$$$$$|
          | $                     *                       #                |
          | $                     *                       #                |
    120us |$+                     *                       #              +-|
          |$                      *                       #                |
          |$                      *                  #   #                 |
    100us |$+         ************########################               +-|
          |$          *          *#                                        |
          |$      *****   #########                                        |
     80us |$+     *    # ####   ##                                       +-|
          |$   **** ### # #                                                |
          |  ** ####                     FBC_LLC_READ_CTRL: 0x8000 ******* |
     60us |-######                       FBC_LLC_READ_CTRL: 0xffff #######-|
          |##        +          +    FBC_LLC_READ_CTRL: 0x400000ff $$$$$$$ |
          +----------------------------------------------------------------+
         0ms       10ms       20ms       30ms      40ms       50ms       60ms
                                   sleep duration

The default FBC_LLC_READ_CTRL value of 0xff is documented to give us
a 170usec delay. That tracks well with the knees at 0xffff->~44msec and
0x8000->~22msec we see in the graph.

We can see that if we sleep longer than the FBC_LLC_READ_CTRL delay
we always observe the full (~145usec) rc6 wakeup latency. But if we sleep
for less than the FBC_LLC_READ_CTRL delay we see a quicker fw wakeup,
presumably due the hardware not having yet entered rc6 fully.
The other plateaus in the graph I suspect correspond to some shallower
internal rc states.

v2: s/usec/msec/ typo in commit msg
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200716190426.17047-2-ville.syrjala@linux.intel.comReviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

DG1 shares some workarounds with TGL and RKL and also has some
additional workarounds of its own.

v2: Correct location of Wa_1408615072 (JohnH).
v3: Apply WAs 1606700617, 18011464164 and 22010931296 to DG1 (José)
v4 (Anusha)
  - Add Wa_22010271021
  - s/Wa_14010096844/Wa_1409836686
v5:
  - Extend Wa_14010919138 to all revs (Matt Atwood)
  - Power gate media is global gen12 design. (Rodrigo)
  - Rebase (Lucas)
v6: use REG_BIT() to fix checkpatch warning (Lucas)

BSpec: 53508

Cc: Matt Atwood <matthew.s.atwood@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Cc: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: NStuart Summers <stuart.summers@intel.com>
Signed-off-by: NAnusha Srivatsa <anusha.srivatsa@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: NLucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: NLucas De Marchi <lucas.demarchi@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20201014191937.1266226-8-lucas.demarchi@intel.com

intel_{enable,disable}_sagv() are no longer needed outside the
compilation unit. Make them static.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200925121749.708-1-ville.syrjala@linux.intel.comReviewed-by: NJosé Roberto de Souza <jose.souza@intel.com>

TGL made stepping a litte mess, workarounds refer to the stepping of
the IP(GT or Display) not of the GPU stepping so it would already
require the same solution as used in commit 96c5a15f
("drm/i915/kbl: Fix revision ID checks").
But to make things even more messy it have a different IP stepping
mapping between SKUs and the same stepping revision of GT do not match
the same HW between TGL U/Y and regular TGL.

So it was required to have 2 different macros to check GT WAs while
for Display we are able to use just one macro that uses the right
revids table.

All TGL workarounds checked and updated accordingly.

v2:
- removed TODO to check if WA 14010919138 applies to regular TGL.
- fixed display stepping in regular TGL (Anusha)

BSpec: 52890
BSpec: 55378
BSpec: 44455
Reviewed-by: NAnusha Srivatsa <anusha.srivtsa@intel.com>
Cc: Anusha Srivatsa <anusha.srivatsa@intel.com>
Cc: Penne Lee <penne.y.lee@intel.com>
Cc: Guangyao Bai <guangyao.bai@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: NJosé Roberto de Souza <jose.souza@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200827233943.400946-1-jose.souza@intel.com

We usually assume that increasing PCI device revision ID's translates to
newer steppings; macros like IS_KBL_REVID() that we use rely on this
behavior.  Unfortunately this turns out to not be true on KBL; the
newer device 2 revision ID's sometimes go backward to older steppings.
The situation is further complicated by different GT and display
steppings associated with each revision ID.

Let's work around this by providing a table to map the revision ID to
specific GT and display steppings, and then perform our comparisons on
the mapped values.

v2:
 - Move the kbl_revids[] array to intel_workarounds.c to avoid compiler
   warnings about an unused variable in files that don't call the
   macros (kernel test robot).

Bspec: 18329
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200811032105.2819370-1-matthew.d.roper@intel.comReviewed-by: NSwathi Dhanavanthri <swathi.dhanavanthri@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>

It's silly to have if(SKL) checks in gen9_init_clock_gating() when
we can just move those bits into skl_init_clock_gating().

I'm not entirely convinced we even need this w/a, or if we do
then maybe we want it for kbl/cfl as well. IIRC it was only
listed in the wadb, but that is now dead so can't double check
anymore. Bspec doesn't seem to have any purely skl specific
DOP clock gating workarounds listed.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200716190426.17047-1-ville.syrjala@linux.intel.comReviewed-by: NJosé Roberto de Souza <jose.souza@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>

Pimp the comments for the FBC related workarounds.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200708131223.9519-4-ville.syrjala@linux.intel.comReviewed-by: NJosé Roberto de Souza <jose.souza@intel.com>