Adds framework for Broxton HW WAs
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

drivers/gpu/drm/i915/intel_ringbuffer.c:435:1-4: WARNING: end returns can be simpified

 Simplify a trivial if-return sequence.  Possibly combine with a
 preceding function call.
Generated by: scripts/coccinelle/misc/simple_return.cocci

CC: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NFengguang Wu <fengguang.wu@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Kill the blt/render tracking we currently have and use the frontbuffer
tracking infrastructure.

Don't enable things by default yet.

v2: (Rodrigo) Fix small conflict on rebase and typo at subject.
v3: (Paulo) Rebase on RENDER_CS change.
v4: (Paulo) Rebase.
v5: (Paulo) Simplify: flushes don't have origin (Daniel).
            Also rebase due to patch order changes.
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In execlist mode, the ringbuf is a function of the ring and context whereas in
legacy mode, it is derived from the ring alone. Thus the calculation required to
determine the ringbuf pointer from the ring (and context) also needs to test
execlist mode or not. This is messy.

Further, the request structure holds a pointer to both the ring and the context
for which it was created. Thus, given a request, it is possible to derive the
ringbuf in either legacy or execlist mode. Hence it is necessary to pass just
the request in to all the low level functions rather than some combination of
request, ring, context and ringbuf. However, rather than recalculating it each
time, it is much simpler to just cache the ringbuf pointer in the request
structure itself.

Caching the pointer means the calculation is done once at request creation time
and all further code and simply read it directly from the request structure.

OTC-Jira: VIZ-5115
Signed-off-by: NJohn Harrison <John.C.Harrison@Intel.com>
[danvet: Drop contentless comment in lrc alloc request entirely. And
spelling fix in the commit message.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

There is a flags word that is passed through the execbuffer code path all the
way from initial decoding of the user parameters down to the very final dispatch
buffer call. It is simply called 'flags'. Unfortuantely, there are many other
flags words floating around in the same blocks of code. Even more once the GPU
scheduler arrives.

This patch makes it more obvious exactly which flags word is which by renaming
'flags' to 'dispatch_flags'. Note that the bit definitions for this flags word
already have an 'I915_DISPATCH_' prefix on them and so are not quite so
ambiguous.

OTC-Jira: VIZ-1587
Signed-off-by: NJohn Harrison <John.C.Harrison@Intel.com>
[danvet: Resolve conflict with Chris' rework of the bb parsing.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

When one EU is disabled in a particular subslice, we can tune how the
work is spread between subslices to improve EU utilization.

v2: - Use a bitfield to record which subslice(s) has(have) 7 EUs. That
      will also make the machinery work if several sublices have 7 EUs.
      (Jeff Mcgee)
    - Only apply the different hashing algorithm if the slice is
      effectively unbalanced by checking there's a single subslice with
      7 EUs. (Jeff Mcgee)

v3: Fix typo in comment (Jeff Mcgee)

Issue: VIZ-3845
Cc: Jeff Mcgee <jeff.mcgee@intel.com>
Reviewed-by: NJeff Mcgee <jeff.mcgee@intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

I have no idea how that crept in, but we need to do the write from the
ring and this is a masked register. Two fixes in 1!

Cc: Nick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It's always a good idea to keep static analysis happy (also because it
prompts doing the check like I proposed :), this time smatch complains:

drivers/gpu/drm/i915/intel_ringbuffer.c:891 gen9_init_workarounds() warn:
  always true condition '((->dev->pdev->revision) >= (0)) => (0-255 >= 0)'

That's because revision is a u8. Tweak a bit the condition then.

Cc: Nick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This function will host SKL-only W/As.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NNick Hoath <nicholas.hoath@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This function is only used in intel_ringbuffer.c, so restrict it to that
file. The function was moved around to avoid a forward declaration and
group it with its user.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
[danvet: Squash in fixup from Wu Fengguang.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2: Reorder defines (Ben)
v3: More bikesheds, this time re-ordering comments! (Chris)
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
[danvet: Resolve conflict.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Fixed the stepping check on WaDisableDgMirrorFixInHalfSliceChicken5
to be for the correct SOC (Skylake)
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2: Don't add WaHdcDisableFetchWhenMasked. Add stepping check for
WaForceEnableNonCoherent
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Move Wa4x4STCOptimizationDisable to gen9_init_workarounds

v2: rebase
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Move WaEnableYV12BugFixInHalfSliceChicken7 to gen9_init_workarounds

v2: Add stepping check.
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This one doesn't have one of these nice cryptic names unfortunately.

v2: Added missing register bitmap
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Move WaDisableDgMirrorFixInHalfSliceChicken5 to gen9_init_workarounds

v2: Added stepping check

v3: Removed unused register bitmap
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
[danvet: Bikesheds.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2: Dont add WaDisableThreadStallDopClockGating as not SKL WA. (Found
by Damien Lespiau)
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
[danvet: Bikeshed commit message a bit as per Damien's suggestions.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Add framework for gen 9 HW WAs

v1: Changed SOC specific WA function to gen 9 common function (Req: Damien Lespiau)
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We already track this in the intel_info struct.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
[danvet: Make the commit message a bit less terse.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This looked like an odd regression from

commit ec5cc0f9
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date:   Thu Jun 12 10:28:55 2014 +0100

    drm/i915: Restrict GPU boost to the RCS engine

but in reality it undercovered a much older coherency bug. The issue that
boosting the GPU frequency on the BCS ring was masking was that we could
wake the CPU up after completion of a BCS batch and inspect memory prior
to the write cache being fully evicted. In order to serialise the
breadcrumb interrupt (and so ensure that the CPU's view of memory is
coherent) we need to perform a post-sync operation in the MI_FLUSH_DW.

v2: Fix all the MI_FLUSH_DW (bsd plus the duplication in execlists).

Also fix the invalidate_domains mask in gen8_emit_flush() for ring !=
VCS.

Testcase: gpuX-rcs-gpu-read-after-write
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: stable@vger.kernel.org
Acked-by: NDaniel Vetter <daniel@ffwll.ch>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

I ran a few tests with xonotic and synmark2 trying out the
different WIZ hashing modes on CHV. The results seem to match the
results I got with IVB/HSW when I did the similar tests on them
in the past. That is 16x4 is generally the fastest mode, 8x8 comes
next and finally 8x4. On CHV the difference between the modes is
at most ~1% in most tests. IIRC on IVB/HSW the difference was a little
bigger, but as there doesn't seem to be any real downside to 16x4
let's use it by default.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NArun Siluvery <arun.siluvery@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Wa4x4STCOptimizationDisable got only implemented for BDW, but according
to the w/a database CHV needs it too, so add it.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NArun Siluvery <arun.siluvery@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We have multiple forcewake domains now on recent gens. Change the
function naming to reflect this.

v2: More verbose names (Chris)
v3: Rebase
v4: Rebase
v5: Add documentation for forcewake_get/put
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: Deepak S <deepak.s@linux.intel.com> (v2)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Where there were duplicate variables for the tail, context and ring (engine)
in the gem request and the execlist queue item, use the one from the request
and remove the duplicate from the execlist queue item.

Issue: VIZ-4274

v1: Rebase
v2: Fixed build issues. Keep separate postfix & tail pointers as these are
used in different ways. Reinserted missing full tail pointer update.
Signed-off-by: NNick Hoath <nicholas.hoath@intel.com>
Reviewed-by: NThomas Daniel <thomas.daniel@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This is an important optimization for avoiding read-after-write (RAW)
stalls in the HiZ buffer.  Certain workloads would run very slowly with
HiZ enabled, but run much faster with the "hiz=false" driconf option.
With this patch, they run at full speed even with HiZ.

Increases performance in OglVSInstancing by about 2.7x on Braswell.
Signed-off-by: NKenneth Graunke <kenneth@whitecape.org>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This is an important optimization for avoiding read-after-write (RAW)
stalls in the HiZ buffer.  Certain workloads would run very slowly with
HiZ enabled, but run much faster with the "hiz=false" driconf option.
With this patch, they run at full speed even with HiZ.

Improves performance in OglVSInstancing by 3.2x on Broadwell GT3e
(Iris Pro 6200).

Thanks to Jesse Barnes and Ben Widawsky for their help in tracking this
down.  Thanks to Chris Wilson for showing me the new workarounds system.
Signed-off-by: NKenneth Graunke <kenneth@whitecape.org>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Found by reading the HIZ_CHICKEN documentation.

Improves performance in a HiZ microbenchmark by around 50%.
Improves performance in OglZBuffer by around 18%.

Thanks to Chris Wilson for helping me figure out where to put this.
Signed-off-by: NKenneth Graunke <kenneth@whitecape.org>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In order to act as a full command barrier by itself, we need to tell the
pipecontrol to actually stall the command streamer while the flush runs.
We require the full command barrier before operations like
MI_SET_CONTEXT, which currently rely on a prior invalidate flush.

References: https://bugs.freedesktop.org/show_bug.cgi?id=83677
Cc: Simon Farnsworth <simon@farnz.org.uk>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: stable@vger.kernel.org
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

In the gen7 pipe control there is an extra bit to flush the media
caches, so let's set it during cache invalidation flushes.

v2: Rename to MEDIA_STATE_CLEAR to be more inline with spec.

Cc: Simon Farnsworth <simon@farnz.org.uk>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Cc: stable@vger.kernel.org
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Otherwise, new platforms without workarounds will hit this warning for
every new context created.

Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NMichel Thierry <michel.thierry@intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We already implement this workaround, but it was missing its name.
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We may be hidding bugs by doing that, so let remove it and have the
actual mask value shine through, for better or worse.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

While trying to unify the order of those arguments throughout the
driver, Daniel noticed what we were inverting them in this part of the
code.
Suggested-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

I was playing with clang and oh surprise! a warning trigerred by
-Wshift-overflow (gcc doesn't have this one):

    WA_SET_BIT_MASKED(GEN7_GT_MODE,
                      GEN6_WIZ_HASHING_MASK | GEN6_WIZ_HASHING_16x4);

    drivers/gpu/drm/i915/intel_ringbuffer.c:786:2: warning: signed shift result
      (0x28002000000) requires 43 bits to represent, but 'int' only has 32 bits
      [-Wshift-overflow]
        WA_SET_BIT_MASKED(GEN7_GT_MODE,
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    drivers/gpu/drm/i915/intel_ringbuffer.c:737:15: note: expanded from macro
      'WA_SET_BIT_MASKED'
        WA_REG(addr, _MASKED_BIT_ENABLE(mask), (mask) & 0xffff)

Turned out GEN6_WIZ_HASHING_MASK was already shifted by 16, and we were
trying to shift it a bit more.

The other thing is that it's not the usual case of setting WA bits here, we
need to have separate mask and value.

To fix this, I've introduced a new _MASKED_FIELD() macro that takes both the
(unshifted) mask and the desired value and the rest of the patch ripples
through from it.

This bug was introduced when reworking the WA emission in:

  Commit 7225342a
  Author: Mika Kuoppala <mika.kuoppala@linux.intel.com>
  Date:   Tue Oct 7 17:21:26 2014 +0300

      drm/i915: Build workaround list in ring initialization

v2: Invert the order of the mask and value arguments (Daniel Vetter)
    Rewrite _MASKED_BIT_ENABLE() and _MASKED_BIT_DISABLE() with
    _MASKED_FIELD() (Jani Nikula)
    Make sure we only evaluate 'a' once in _MASKED_BIT_ENABLE() (Dave Gordon)
    Add check to ensure the value is within the mask boundaries (Chris Wilson)

v3: Ensure the the value and mask are 16 bits (Dave Gordon)

Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: Arun Siluvery <arun.siluvery@linux.intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>