Instead of doing what we do currently, which will never work with
PROVE_LOCKING, do the same as AMD does, and something similar to
relocation slowpath. When all locks are dropped, we acquire the
pages for pinning. When the locks are taken, we transfer those
pages in .get_pages() to the bo. As a final check before installing
the fences, we ensure that the mmu notifier was not called; if it is,
we return -EAGAIN to userspace to signal it has to start over.

Changes since v1:
- Unbinding is done in submit_init only. submit_begin() removed.
- MMU_NOTFIER -> MMU_NOTIFIER
Changes since v2:
- Make i915->mm.notifier a spinlock.
Changes since v3:
- Add WARN_ON if there are any page references left, should have been 0.
- Return 0 on success in submit_init(), bug from spinlock conversion.
- Release pvec outside of notifier_lock (Thomas).
Changes since v4:
- Mention why we're clearing eb->[i + 1].vma in the code. (Thomas)
- Actually check all invalidations in eb_move_to_gpu. (Thomas)
- Do not wait when process is exiting to fix gem_ctx_persistence.userptr.
Changes since v5:
- Clarify why check on PF_EXITING is (temporarily) required.
Changes since v6:
- Ensure userptr validity is checked in set_domain through a special path.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Acked-by: NDave Airlie <airlied@redhat.com>
[danvet: s/kfree/kvfree/ in i915_gem_object_userptr_drop_ref in the
previous review round, but which got lost. The other open questions
around page refcount are imo better discussed in a separate series,
with amdgpu folks involved].
Reviewed-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-17-maarten.lankhorst@linux.intel.com

Now that unsynchronized mappings are removed, the only time userptr
works is when the MMU notifier is enabled. Put all of the userptr
code behind a mmu notifier ifdef.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-16-maarten.lankhorst@linux.intel.com

We should not allow this any more, as it will break with the new userptr
implementation, it could still be made to work, but there's no point in
doing so.

Inspection of the beignet opencl driver shows that it's only used
when normal userptr is not available, which means for new kernels
you will need CONFIG_I915_USERPTR.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Acked-by: NDave Airlie <airlied@redhat.com>
Reviewed-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Acked-by: NJason Ekstrand <jason@jlekstrand.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-15-maarten.lankhorst@linux.intel.com

There are a couple of ioctl's related to tiling and cache placement,
that make no sense for userptr, reject those:
- i915_gem_set_tiling_ioctl()
    Tiling should always be linear for userptr. Changing placement will
    fail with -ENXIO.
- i915_gem_set_caching_ioctl()
    Userptr memory should always be cached. Changing caching mode will
    fail with -ENXIO.
- i915_gem_set_domain_ioctl()
    Still temporarily allowed to work as intended, it's used to check
    userptr validity. With the reworked userptr code, it will keep
    working for this usecase.

This plus the previous changes have been tested against beignet
by using its own unit tests, and intel-video-compute by using
piglit's opencl tests.

Changes since v1:
- set_domain was apparently used in iris for checking userptr validity,
  keep it working as intended.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Acked-by: NJason Ekstrand <jason@jlekstrand.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-14-maarten.lankhorst@linux.intel.com

It doesn't make sense to export a memory address, we will prevent
allowing access this way to different address spaces when we
rework userptr handling, so best to explicitly disable it.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Acked-by: NJason Ekstrand <jason@jlekstrand.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-13-maarten.lankhorst@linux.intel.com

Userptr should not need the kernel for a userspace memcpy, userspace
needs to call memcpy directly.

Specifically, disable i915_gem_pwrite_ioctl() and i915_gem_pread_ioctl().
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Acked-by: NJason Ekstrand <jason@jlekstrand.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-12-maarten.lankhorst@linux.intel.com

As soon as we install fences, we should stop allocating memory
in order to prevent any potential deadlocks.

This is required later on, when we start adding support for
dma-fence annotations.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-11-maarten.lankhorst@linux.intel.com

Simple adding of i915_gem_object_lock, we may start to pass ww to
get_pages() in the future, but that won't be the case here;
We override shmem's get_pages() handling by calling
i915_gem_object_get_pages_phys(), no ww is needed.

Changes since v1:
- Call shmem put pages directly, the callback would
  go down the phys free path.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-10-maarten.lankhorst@linux.intel.com

Instead of creating a separate object type, we make changes to
the shmem type, to clear struct page backing. This will allow us to
ensure we never run into a race when we exchange obj->ops with other
function pointers.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-9-maarten.lankhorst@linux.intel.com

We want to remove the changing of ops structure for attaching
phys pages, so we need to kill off HAS_STRUCT_PAGE from ops->flags,
and put it in the bo.

This will remove a potential race of dereferencing the wrong obj->ops
without ww mutex held.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
[danvet: apply with wiggle]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-8-maarten.lankhorst@linux.intel.com

Doesn't need the full ww lock, only checking if pages are bound.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> #irc
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-7-maarten.lankhorst@linux.intel.com

Currently we have a lot of places where we hold the gem object lock,
but haven't yet been converted to the ww dance. Complain loudly about
those places.

i915_vma_pin shouldn't have the obj lock held, so we can do a ww dance,
while i915_vma_pin_ww should.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> #irc
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-6-maarten.lankhorst@linux.intel.com

i915_vma_pin may fail with -EDEADLK when we start locking page tables,
so ensure we handle this correctly.

Changes since v1:
- Drop -EDEADLK todo, this commit handles it.
- Change eb_pin_vma from sort-of-bool + -EDEADLK to a proper int. (Matt)

Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-5-maarten.lankhorst@linux.intel.com

We need to get rid of allocations in the cmd parser, because it needs
to be called from a signaling context, first move all pinning to
execbuf, where we already hold all locks.

Allocate jump_whitelist in the execbuffer, and add annotations around
intel_engine_cmd_parser(), to ensure we only call the command parser
without allocating any memory, or taking any locks we're not supposed to.

Because i915_gem_object_get_page() may also allocate memory, add a
path to i915_gem_object_get_sg() that prevents memory allocations,
and walk the sg list manually. It should be similarly fast.

This has the added benefit of being able to catch all memory allocation
errors before the point of no return, and return -ENOMEM safely to the
execbuf submitter.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Acked-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-4-maarten.lankhorst@linux.intel.com

We're starting to require the reservation lock for pinning,
so wait until we have that.

Update the selftests to handle this correctly, and ensure pin is
called in live_hwsp_rollover_user() and mock_hwsp_freelist().

Changes since v1:
- Fix NULL + XX arithmatic, use casts. (kbuild)
Changes since v2:
- Clear entire cacheline when pinning.
Changes since v3:
- CACHELINE_BYTES -> TIMELINE_SEQNO_BYTES. (jekstrand)
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reported-by: Nkernel test robot <lkp@intel.com>
Reviewed-by: NThomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-3-maarten.lankhorst@linux.intel.com

Instead of sharing pages with breadcrumbs, give each timeline a
single page. This allows unrelated timelines not to share locks
any more during command submission.

As an additional benefit, seqno wraparound no longer requires
i915_vma_pin, which means we no longer need to worry about a
potential -EDEADLK at a point where we are ready to submit.

Changes since v1:
- Fix erroneous i915_vma_acquire that should be a i915_vma_release (ickle).
- Extra check for completion in intel_read_hwsp().
Changes since v2:
- Fix inconsistent indent in hwsp_alloc() (kbuild)
- memset entire cacheline to 0.
Changes since v3:
- Do same in intel_timeline_reset_seqno(), and clflush for good measure.
Changes since v4:
- Use refcounting on timeline, instead of relying on i915_active.
- Fix waiting on kernel requests.
Changes since v5:
- Bump amount of slots to maximum (256), for best wraparounds.
- Add hwsp_offset to i915_request to fix potential wraparound hang.
- Ensure timeline wrap test works with the changes.
- Assign hwsp in intel_timeline_read_hwsp() within the rcu lock to
  fix a hang.
Changes since v6:
- Rename i915_request_active_offset to i915_request_active_seqno(),
  and elaborate the function. (tvrtko)
Changes since v7:
- Move hunk to where it belongs. (jekstrand)
- Replace CACHELINE_BYTES with TIMELINE_SEQNO_BYTES. (jekstrand)
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@intel.com> #v1
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323155059.628690-2-maarten.lankhorst@linux.intel.com

Use i915 prefix to avoid name collision with future vma_lookup() in mm.
Signed-off-by: NLiam R. Howlett <Liam.Howlett@Oracle.com>
Reviewed-by: NMatthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210323134208.3077275-1-Liam.Howlett@Oracle.com

The rationale for this change is roughly as follows:

 1. The functionality can be done entirely in userspace with a
    combination of mmap + memcpy

 2. The only reason anyone in userspace is still using it is because
    someone implemented bo_subdata that way in libdrm ages ago and
    they're all too lazy to write the 5 lines of code to do a map.

 3. This falls cleanly into the category of things which will only get
    more painful with local memory support.

These ioctls aren't used much anymore by "real" userspace drivers.
Vulkan has never used them and neither has the iris GL driver.  The old
i965 GL driver does use PWRITE for glBufferSubData but it only supports
up through Gen11; Gen12 was never enabled in i965.  The compute driver
has never used PREAD/PWRITE.  The only remaining user is the media
driver which uses it exactly twice and they're easily removed [1] so
expecting them to drop it going forward is reasonable.

IGT changes which handle this kernel change have also been submitted [2].

[1] https://github.com/intel/media-driver/pull/1160
[2] https://patchwork.freedesktop.org/series/81384/

v2 (Jason Ekstrand):
 - Improved commit message with the status of all usermode drivers
 - A more future-proof platform check

v3 (Jason Ekstrand):
 - Drop the HAS_LMEM checks as they're already covered by the version
   checks
Signed-off-by: NAshutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: NJason Ekstrand <jason@jlekstrand.net>
Reviewed-by: NJason Ekstrand <jason@jlekstrand.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210317234014.2271006-4-jason@jlekstrand.net

The Vulkan driver in Mesa for Intel hardware never uses relocations if
it's running on a version of i915 that supports at least softpin which
all versions of i915 supporting Gen12 do.  On the OpenGL side, Gen12+ is
only supported by iris which never uses relocations.  The older i965
driver in Mesa does use relocations but it only supports Intel hardware
through Gen11 and has been deprecated for all hardware Gen9+.  The
compute driver also never uses relocations.  This only leaves the media
driver which is supposed to be switching to softpin going forward.
Making softpin a requirement for all future hardware seems reasonable.

There is one piece of hardware enabled by default in i915: RKL which was
enabled by e22fa6f0 which has not yet landed in drm-next so this
almost but not really a userspace API change for RKL.  If it becomes a
problem, we can always add !IS_ROCKETLAKE(eb->i915) to the condition.

Rejecting relocations starting with newer Gen12 platforms has the
benefit that we don't have to bother supporting it on platforms with
local memory.  Given how much CPU touching of memory is required for
relocations, not having to do so on platforms where not all memory is
directly CPU-accessible carries significant advantages.

v2 (Jason Ekstrand):
 - Allow TGL-LP platforms as they've already shipped

v3 (Jason Ekstrand):
 - WARN_ON platforms with LMEM support in case the check is wrong

v4 (Jason Ekstrand):
 - Call out Rocket Lake in the commit message

v5 (Jason Ekstrand):
 - Drop the HAS_LMEM check as it's already covered by the version check

v6 (Jason Ekstrand):
 - Move the check to eb_validate_vma() with all the other exec_object
   validation checks.
Signed-off-by: NJason Ekstrand <jason@jlekstrand.net>
Reviewed-by: NZbigniew Kempczyński <zbigniew.kempczynski@intel.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210317234014.2271006-3-jason@jlekstrand.net

libdrm has supported the newer execbuffer2 ioctl and using it by default
when it exists since libdrm commit b50964027bef which landed Mar 2, 2010.
The i915 and i965 drivers in Mesa at the time both used libdrm and so
did the Intel X11 back-end.  The SNA back-end for X11 has always used
execbuffer2.

v2 (Jason Ekstrand):
 - Add a comment saying what Linux version it's being removed in.
Signed-off-by: NJason Ekstrand <jason@jlekstrand.net>
Acked-by: NKeith Packard <keithp@keithp.com>
Acked-by: NDave Airlie <airlied@redhat.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20210317234014.2271006-2-jason@jlekstrand.net

One instance of DRM_DEBUG_KMS was leftover in dp_link_training, convert
it to the new shiny.
Signed-off-by: NSean Paul <seanpaul@chromium.org>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210310214845.29021-2-sean@poorly.run

This patch adds some newlines which are missing from debug messages.
This will prevent logs from being stacked up in dmesg.
Signed-off-by: NSean Paul <seanpaul@chromium.org>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210310214845.29021-1-sean@poorly.run

SAMPLE_OA parameter enables sampling of OA buffer and results in a call
to init the OA buffer which initializes the OA unit head/tail pointers.
The OA_EXPONENT parameter controls the periodicity of the OA reports in
the OA buffer and results in starting a hrtimer.

Before gen12, all use cases required the use of the OA buffer and i915
enforced this setting when vetting out the parameters passed. In these
platforms the hrtimer was enabled if OA_EXPONENT was passed. This worked
fine since it was implied that SAMPLE_OA is always passed.

With gen12, this changed. Users can use perf without enabling the OA
buffer as in OAR use cases. While an OAR use case should ideally not
start the hrtimer, we see that passing an OA_EXPONENT parameter will
start the hrtimer even though SAMPLE_OA is not specified. This results
in an uninitialized OA buffer, so the head/tail pointers used to track
the buffer are zero.

This itself does not fail, but if we ran a use-case that SAMPLED the OA
buffer previously, then the OA_TAIL register is still pointing to an old
value. When the timer callback runs, it ends up calculating a
wrong/large number of available reports. Since we do a spinlock_irq_save
and start processing a large number of reports, NMI watchdog fires and
causes a crash.

Start the timer only if SAMPLE_OA is specified.

v2:
- Drop SAMPLE OA check when appending samples (Ashutosh)
- Prevent read if OA buffer is not being sampled

Fixes: 00a7f0d7 ("drm/i915/tgl: Add perf support on TGL")
Signed-off-by: NUmesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Reviewed-by: NAshutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: NLionel Landwerlin <lionel.g.landwerlin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210305210947.58751-1-umesh.nerlige.ramappa@intel.com

Rename a bunch of the skl+ watermark struct members to
have sensible names. Avoids me having to think what
plane_res_b/etc. means.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210305153610.12177-7-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

Make the code more typo proof by extracting small helpers that
do the "do we have enough DDB for the WM level?" checks in
a consistent manner.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210305153610.12177-6-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

Let's make all the "do we have enough DDB for this WM level?"
checks use min_ddb_alloc. To achieve that we need to populate
this for the transition watermarks as well.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210305153610.12177-5-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

For non-transition watermarks we are supposed to check min_ddb_alloc
rather than plane_res_b when determining if we have enough DDB space
for it. A bit too much copy pasta made me check the wrong thing.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Fixes: df4a50a3 ("drm/i915: Zero out SAGV wm when we don't have enough DDB for it")
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210305153610.12177-4-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

Say we have two planes enabled with watermarks configured
as follows:
plane A: wm0=enabled/can_sagv=false, wm1=enabled/can_sagv=true
plane B: wm0=enabled/can_sagv=true,  wm1=disabled

This is possible since the latency we use to calculate
can_sagv may not be the same for both planes due to
skl_needs_memory_bw_wa().

In this case skl_crtc_can_enable_sagv() will see that
both planes have enabled at least one watermark level
with can_sagv==true, and thus proceeds to allow SAGV.
However, since plane B does not have wm1 enabled
plane A can't actually use it either. Thus we are
now running with SAGV enabled, but plane A can't
actually tolerate the extra latency it imposes.

To remedy this only allow SAGV on if the highest common
enabled watermark level for all active planes can tolerate
the extra SAGV latency.

Cc: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210305153610.12177-3-ville.syrjala@linux.intel.comReviewed-by: NStanislav Lisovskiy <stanislav.lisovskiy@intel.com>

On HSW/BDW with VT-d active the first tile row scanned out
after the first async flip of the frame often ends up corrupted.

Whether the corruption happens or not depends on the scanline
on which the async flip happens, but the behaviour seems very
consistent. Ie. the same set of scanlines (which are most scanlines)
always show the corruption. And another set of scanlines (far less
of them) never shows the corruption.

I discovered that disabling the fetch-stride stretching
feature cures the corruption. This is some kind of TLB related
prefetch thing AFAIK. We already disable it on SNB primary
planes due to a documented workaround. The hardware folks
indicated that disabling this should be fine, so let's go
with that.

And while we're here, let's document the relevant bits on all
pre-skl platforms.

Fixes: 2a636e24 ("drm/i915: Implement async flip for ivb/hsw")
Fixes: cda195f1 ("drm/i915: Implement async flips for bdw")
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210220103303.3448-1-ville.syrjala@linux.intel.comReviewed-by: NKarthik B S <karthik.b.s@intel.com>

Commit 311a50e7 ("drm/i915: Add support for mandatory cmdparsing")
introduced mandatory command parsing but setup failures were not
translated into wedging the GPU which was probably the intent.

Possible errors come in two categories. Either the sanity check on
internal tables has failed, which should be caught in CI unless an
affected platform would be missed in testing; or memory allocation failure
happened during driver load, which should be extremely unlikely but for
correctness should still be handled.

v2:
 * Tidy coding style. (Chris)

[airlied: cherry-picked to avoid rc1 base]
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Fixes: 311a50e7 ("drm/i915: Add support for mandatory cmdparsing")
Cc: Jon Bloomfield <jon.bloomfield@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Chris Wilson <chris.p.wilson@intel.com>
Reviewed-by: NChris Wilson <chris.p.wilson@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210302114213.1102223-1-tvrtko.ursulin@linux.intel.com
(cherry picked from commit 5a1a659762d35a6dc51047c9127c011303c77b7f)
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Let's check that we actually found the PLL before doing the
port_clock readout, just in case the hardware was severly
misprogrammed by the previous guy. Not sure the hw would
even survive such misprogramming without hanging but no
real harm in checking anyway.

Cc: Karthik B S <karthik.b.s@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210310194351.6233-1-ville.syrjala@linux.intel.comReviewed-by: NKarthik B S <karthik.b.s@intel.com>

Now that all the encoder clock stuff is uniformly abstracted
for all hsw+ platforms, let's extend icl_sanitize_encoder_pll_mapping()
to cover all of them.

Not sure there is a particular benefit in doing so, but less special
cases always makes me happy.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210224144214.24803-7-ville.syrjala@linux.intel.comReviewed-by: NMika Kahola <mika.kahola@intel.com>

Support reading out the current state of the DDI clock.

Not sure we really want this. Seems a bit excessive just to
restore the debug print to icl_sanitize_encoder_pll_mapping()?
But maybe there's more use for it?
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210224144214.24803-6-ville.syrjala@linux.intel.comReviewed-by: NMika Kahola <mika.kahola@intel.com>

Move the *_get_ddi_pll() stuff into the encodet->get_config() hook.
There it neatly sits next to the matching .{enable,disable}_clock()
functions.

In order to avoid excessive boilerplate I changed the behaviour
such that all platforms now do the readout via
crtc_state->port_dpll[].

ICL+ TC is still a bit special due to TBTPLL not having a functional
.get_freq(). Should probably change that by adopting the LCPLL
approach, but that would require a fairly substantial rework of the
DPLL ID handling. So leave it for later.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210224144214.24803-5-ville.syrjala@linux.intel.comReviewed-by: NMika Kahola <mika.kahola@intel.com>

All the other places we have use pipes instead of crtc indices
when tracking resource usage. Life is easier when we do it
the same way always, so switch the dpll mgr to using pipes as
well. Looks like it was actually mixing these up in some cases
so it would not even have worked correctly except when the
device has a contiguous set of pipes starting from pipe A.
Granted, that is the typical case but supposedly it may not
always hold on modern hw.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210224144214.24803-4-ville.syrjala@linux.intel.comReviewed-by: NMika Kahola <mika.kahola@intel.com>

The clock readout for DDI encoders needs to moved into the encoders.
To that end intel_dpll_readout_hw_state() needs to happen after
the encoder readout as otherwise it can't correctly populate
the PLL crtc_mask/active_mask bitmasks.

v2: Populate DPLL ref clocks before the encoder->get_config()
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210225161225.30746-1-ville.syrjala@linux.intel.comReviewed-by: NMika Kahola <mika.kahola@intel.com>

Stop assuming intel_ddi_get_config() is all we need from the primary
encoder, and instead call it via the .get_config() vfunc. This
will allow customized .get_config() for the primary, which I plan
to use to handle the differences in the clock readout between various
platforms.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210224144214.24803-2-ville.syrjala@linux.intel.comReviewed-by: NMika Kahola <mika.kahola@intel.com>

commit 33267703 ("drm/i915/dsi: Enable software vblank counter")
claims to get the mode_flags from the crtc_state, but in fact does
not. Fix it to do it right.

Cc: Vandita Kulkarni <vandita.kulkarni@intel.com>
Cc: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210304170421.10901-5-ville.syrjala@linux.intel.comReviewed-by: NJani Nikula <jani.nikula@intel.com>

We print the scanline counters as unsigned integers so the -1
here just makes the debugs/traces look a bit messy. Zero seems
equally valid for this usecase.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210304170421.10901-4-ville.syrjala@linux.intel.comReviewed-by: NJani Nikula <jani.nikula@intel.com>

For platforms/outputs without hardware frame counters we can't
call drm_crtc_accurate_vblank_count() when the vblank support is
disabled or we just get a WARN due to the crtc timings
(vblank->hwmode) being considered invalid. Note that until the
pipe in question has been enabled and drm_crtc_set_max_vblank_count()
has been called on it we would also take this path on platforms
which have a working frame counter. So getting the WARN is rather
likely on any platform unless you always boot with lots of displays
plugged in.

Also even on hardware with a working frame counter we may not be
able to read the actual frame counter register on disabled pipes
due the relevant power well being disabled. Ie. would just result
in the unclaimed reg spew.

So let's just avoid all this an directly report zero in case
the pipe is disabled.
Reported-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>
Tested-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20210304170421.10901-3-ville.syrjala@linux.intel.comSigned-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>