We use drm_crtc_ for all the new-style vblank functions which directly
take a struct drm_crtc *. drm_accurate_vblank_count was the odd one
out, correct this to appease my OCD.
Acked-by: NThierry Reding <treding@nvidia.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170524145212.27837-13-daniel.vetter@ffwll.ch

In the previous patch we've implemented hwmode tracking a la i915 for
the vblank timestamp calculations. But that was just the basic
semantics, i915 has some nice sanity checks to make sure we keep
getting this right. Move them over too.

v2:
- WARN_ON_ONCE to avoid excessive spam (Ville)
- Really only WARN on atomic drivers.

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NNeil Armstrong <narmstrong@baylibre.com>
Acked-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170509140329.24114-5-daniel.vetter@ffwll.ch

If we restrict this helper to only kms drivers (which is the case) we
can look up the correct mode easily ourselves. But it's a bit tricky:

- All legacy drivers look at crtc->hwmode. But that is updated already
  at the beginning of the modeset helper, which means when we disable
  a pipe. Hence the final timestamps might be a bit off. But since
  this is an existing bug I'm not going to change it, but just try to
  be bug-for-bug compatible with the current code. This only applies
  to radeon&amdgpu.

- i915 tries to get it perfect by updating crtc->hwmode when the pipe
  is off (i.e. vblank->enabled = false).

- All other atomic drivers look at crtc->state->adjusted_mode. Those
  that look at state->requested_mode simply don't adjust their mode,
  so it's the same. That has two problems: Accessing crtc->state from
  interrupt handling code is unsafe, and it's updated before we shut
  down the pipe. For nonblocking modesets it's even worse.

For atomic drivers try to implement what i915 does. To do that we add
a new hwmode field to the vblank structure, and update it from
drm_calc_timestamping_constants(). For atomic drivers that's called
from the right spot by the helper library already, so all fine. But
for safety let's enforce that.

For legacy driver this function is only called at the end (oh the
fun), which is broken, so again let's not bother and just stay
bug-for-bug compatible.

The  benefit is that we can use drm_calc_vbltimestamp_from_scanoutpos
directly to implement ->get_vblank_timestamp in every driver, deleting
a lot of code.

v2: Completely new approach, trying to mimick the i915 solution.

v3: Fixup kerneldoc.

v4: Drop the WARN_ON to check that the vblank is off, atomic helpers
currently unconditionally call this. Recomputing the same stuff should
be harmless.

v5: Fix typos and move misplaced hunks to the right patches (Neil).

v6: Undo hunk movement (kbuild).

Cc: Mario Kleiner <mario.kleiner@tuebingen.mpg.de>
Cc: Eric Anholt <eric@anholt.net>
Cc: Rob Clark <robdclark@gmail.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: NNeil Armstrong <narmstrong@baylibre.com>
Acked-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170509140329.24114-4-daniel.vetter@ffwll.ch

It's overkill to have a flag parameter which is essentially used just
as a boolean. This takes care of core + adjusting drivers.

Adjusting the scanout position callback is a bit harder, since radeon
also supplies it's own driver-private flags in there.

v2: Fixup misplaced hunks (Neil).

v3: kbuild says v1 was better ...

Cc: Mario Kleiner <mario.kleiner@tuebingen.mpg.de>
Cc: Eric Anholt <eric@anholt.net>
Cc: Rob Clark <robdclark@gmail.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NNeil Armstrong <narmstrong@baylibre.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170509140329.24114-2-daniel.vetter@ffwll.ch

There's really no reason for anything more:
- Calling this while the crtc vblank stuff isn't set up is a driver
  bug. Those places alrready DRM_ERROR.
- Calling this when the crtc is off is either a driver bug (calling
  drm_crtc_handle_vblank at the wrong time) or a core bug (for
  anything else). Again, we DRM_ERROR.
- EINVAL is checked at higher levels already, and if we'd use struct
  drm_crtc * instead of (dev, pipe) it would be real obvious that
  those are again core bugs.

The only valid failure mode is crap hardware that couldn't sample a
useful timestamp, to ask the core to just grab a not-so-accurate
timestamp. Bool is perfectly fine for that.

v2: Also fix up the one caller, I lost that in the shuffling (Jani).

v3: Fixup commit message (Neil).

Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Mario Kleiner <mario.kleiner@tuebingen.mpg.de>
Cc: Eric Anholt <eric@anholt.net>
Cc: Rob Clark <robdclark@gmail.com>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: NNeil Armstrong <narmstrong@baylibre.com>
Acked-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170509140329.24114-1-daniel.vetter@ffwll.ch

Previously with commit a9c1f90c
("drm/i915: Don't mask EI UP interrupt on IVB|SNB") certain,
seemingly unrelated bit (GEN6_PM_RP_UP_EI_EXPIRED) was needed
to be unmasked for IVB and SNB in order to prevent system hang
with chained batchbuffers.

Our CI was seeing incomplete results with tests that used
chained batches and it was found out that HSW needs to have this
same bit unmasked to reliably survive chained batches.

Always unmask GEN6_PM_RP_UP_EI_EXPIRED on Haswell to
prevent system hang with batch chaining.

Testcase: igt/gem_exec_fence/nb-await-default
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100672
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: stable@vger.kernel.org
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Acked-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1492082127-29007-1-git-send-email-mika.kuoppala@intel.com
(cherry picked from commit 3396a273)
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

When initializing the GuC log struct, there is an object we need to
allocate always, since the GuC needs its address at fw load time.
The rest is only needed during runtime, in the sense that we only
create if we actually enable GuC logging. Make that distinction
explicit by subdividing further the intel_guc_log struct.

v2: Call the new struct "runtime", instead of "extras" (Joonas)

v3: Check indent (Joonas)
Signed-off-by: NOscar Mateo <oscar.mateo@intel.com>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Reviewed-by: NMichal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Signed-off-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>

I915_RESET_IN_PROGRESS is being used for both signaling the requirement
to i915_mutex_lock_interruptible() to avoid taking the struct_mutex and
to instruct a waiter (already holding the struct_mutex) to perform the
reset. To allow for a little more coordination, split these two meaning
into a couple of distinct flags. I915_RESET_BACKOFF tells
i915_mutex_lock_interruptible() not to acquire the mutex and
I915_RESET_HANDOFF tells the waiter to call i915_reset().
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Acked-by: NMichel Thierry <michel.thierry@intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170316171305.12972-1-chris@chris-wilson.co.uk

This emulates execlists on top of the GuC in order to defer submission of
requests to the hardware. This deferral allows time for high priority
requests to gazump their way to the head of the queue, however it nerfs
the GuC by converting it back into a simple execlist (where the CPU has
to wake up after every request to feed new commands into the GuC).

v2: Drop hack status - though iirc there is still a lockdep inversion
between fence and engine->timeline->lock (which is impossible as the
nesting only occurs on different fences - hopefully just requires some
judicious lockdep annotation)
v3: Apply lockdep nesting to enabling signaling on the request, using
the pattern we already have in __i915_gem_request_submit();
v4: Replaying requests after a hang also now needs the timeline
spinlock, to disable the interrupts at least
v5: Hold wq lock for completeness, and emit a tracepoint for enabling signal
v6: Reorder interrupt checking for a happier gcc.
v7: Only signal the tasklet after a user-interrupt if using guc scheduling
v8: Restore lost update of rq through the i915_guc_irq_handler (Tvrtko)
v9: Avoid re-initialising the engine->irq_tasklet from inside a reset
v10: Hook up the execlists-style tracepoints
v11: Clear the execlists irq_posted bit after taking over the interrupt/tasklet
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170316125619.6856-1-chris@chris-wilson.co.ukReviewed-by: NMichał Winiarski <michal.winiarski@intel.com>

Set byt rc residency counters high level as chv does by
default. We lose some accuracy on byt but we can do the calculation
without extra hw read on both platforms, as now they behave
identically in this respect.

v2: use ktime
v3: keep comparison u32 (Chris)

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1489592584-10422-1-git-send-email-mika.kuoppala@intel.com

We have used cz timestamp register to gain a reference time wrt
to residency calculations. The residency counts are in cz clk ticks
(333Mhz clock) but for some reason the cz timestamp register gives
100us units. Perhaps for some other usage, the base-ten based values
are easier, but in residency calculations raw units would have been
the easiest.

As there is not much advantage of using base-ten clock through
a more costly punit access, take our reference times directly from
kernel clock.

v2: use ktime (Chris, Ville)

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Check that request has not been signaled before acquiring a reference to
the request for signaling later in the interrupt handler.

The loading of the cacheline (for request->fence.flags) should be "free"
when followed by the locked increment of the request->fence.refcount
(which then sets the cacheline to exclusive mode), i.e. the cost of
test_bit prior to an atomic_inc should be negligible. This should
benefit us when we have a pile of bare breadcrumbs (interrupted execbuf)
where we may get interrupts faster than we can get rid of the
intel_wait, or if the device is too slow to run the bottom-half between
interrupts.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170315210726.12095-5-chris@chris-wilson.co.uk

We take the runtime pm wakelock during i915_handle_error() to ensure
that all paths that reach the error handler keep the device awake during
the hw reads. However, we need to extend that from the reset handler to
include the earlier capture routines.
Reported-by: NAntonio Argenziano <antonio.argenziano@intel.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Michel Thierry <michel.thierry@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170314171840.25706-1-chris@chris-wilson.co.ukReviewed-by: NMichel Thierry <michel.thierry@intel.com>

On Baytrail, we manually calculate busyness over the evaluation interval
to avoid issues with miscaluations with RC6 enabled. However, it turns
out that the DOWN_EI interrupt generator is completely bust - it
operates in two modes, continuous or never. Neither of which are
conducive to good behaviour. Stop unmask the DOWN_EI interrupt and just
compute everything from the UP_EI which does seem to correspond to the
desired interval.

v2: Fixup gen6_rps_pm_mask() as well
v3: Inline vlv_c0_above() to combine the now identical elapsed
calculation for up/down and simplify the threshold testing

Fixes: 43cf3bf0 ("drm/i915: Improved w/a for rps on Baytrail")
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: <stable@vger.kernel.org> # v4.1+
Link: http://patchwork.freedesktop.org/patch/msgid/20170309211232.28878-1-chris@chris-wilson.co.ukReviewed-by: NMika Kuoppala <mika.kuoppala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170313170617.31564-1-chris@chris-wilson.co.uk
(cherry picked from commit e0e8c7cb)
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

In order to prevent accessing the hpd registers outside of the display
power wells, we should refrain from writing to the registers before the
display interrupts are enabled.

[    4.740136] WARNING: CPU: 1 PID: 221 at drivers/gpu/drm/i915/intel_uncore.c:795 __unclaimed_reg_debug+0x44/0x50 [i915]
[    4.740155] Unclaimed read from register 0x1e1110
[    4.740168] Modules linked in: i915(+) intel_gtt drm_kms_helper prime_numbers
[    4.740190] CPU: 1 PID: 221 Comm: systemd-udevd Not tainted 4.10.0-rc6+ #384
[    4.740203] Hardware name:                  /        , BIOS PYBSWCEL.86A.0027.2015.0507.1758 05/07/2015
[    4.740220] Call Trace:
[    4.740236]  dump_stack+0x4d/0x6f
[    4.740251]  __warn+0xc1/0xe0
[    4.740265]  warn_slowpath_fmt+0x4a/0x50
[    4.740281]  ? insert_work+0x77/0xc0
[    4.740355]  ? fwtable_write32+0x90/0x130 [i915]
[    4.740431]  __unclaimed_reg_debug+0x44/0x50 [i915]
[    4.740507]  fwtable_read32+0xd8/0x130 [i915]
[    4.740575]  i915_hpd_irq_setup+0xa5/0x100 [i915]
[    4.740649]  intel_hpd_init+0x68/0x80 [i915]
[    4.740716]  i915_driver_load+0xe19/0x1380 [i915]
[    4.740784]  i915_pci_probe+0x32/0x90 [i915]
[    4.740799]  pci_device_probe+0x8b/0xf0
[    4.740815]  driver_probe_device+0x2b6/0x450
[    4.740828]  __driver_attach+0xda/0xe0
[    4.740841]  ? driver_probe_device+0x450/0x450
[    4.740853]  bus_for_each_dev+0x5b/0x90
[    4.740865]  driver_attach+0x19/0x20
[    4.740878]  bus_add_driver+0x166/0x260
[    4.740892]  driver_register+0x5b/0xd0
[    4.740906]  ? 0xffffffffa0166000
[    4.740920]  __pci_register_driver+0x47/0x50
[    4.740985]  i915_init+0x5c/0x5e [i915]
[    4.740999]  do_one_initcall+0x3e/0x160
[    4.741015]  ? __vunmap+0x7c/0xc0
[    4.741029]  ? kmem_cache_alloc+0xcf/0x120
[    4.741045]  do_init_module+0x55/0x1c4
[    4.741060]  load_module+0x1f3f/0x25b0
[    4.741073]  ? __symbol_put+0x40/0x40
[    4.741086]  ? kernel_read_file+0x100/0x190
[    4.741100]  SYSC_finit_module+0xbc/0xf0
[    4.741112]  SyS_finit_module+0x9/0x10
[    4.741125]  entry_SYSCALL_64_fastpath+0x17/0x98
[    4.741135] RIP: 0033:0x7f8559a140f9
[    4.741145] RSP: 002b:00007fff7509a3e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[    4.741161] RAX: ffffffffffffffda RBX: 00007f855aba02d1 RCX: 00007f8559a140f9
[    4.741172] RDX: 0000000000000000 RSI: 000055b6db0914f0 RDI: 0000000000000011
[    4.741183] RBP: 0000000000020000 R08: 0000000000000000 R09: 000000000000000e
[    4.741193] R10: 0000000000000011 R11: 0000000000000246 R12: 000055b6db0854d0
[    4.741204] R13: 000055b6db091150 R14: 0000000000000000 R15: 000055b6db035924

v2: Set dev_priv->display_irqs_enabled to true for all platforms other
than vlv/chv that manually control the display power domain.

Fixes: 19625e85 ("drm/i915: Enable polling when we don't have hpd")
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=97798Suggested-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Lyude <cpaul@redhat.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Hans de Goede <jwrdegoede@fedoraproject.org>
Cc: stable@vger.kernel.org
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170215131547.5064-1-chris@chris-wilson.co.uk
Link: http://patchwork.freedesktop.org/patch/msgid/20170313170231.18633-1-chris@chris-wilson.co.uk
(cherry picked from commit 262fd485)
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Replace __raw_i915_read32() with I915_READ_FW() in the workaround for
the SKL+ scanline counter hardware fail. The two are the same thing
but everyone else uses I915_READ_FW() so let's follow suit.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170309154434.29303-3-ville.syrjala@linux.intel.comReviewed-by: NMika Kahola <mika.kahola@intel.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>

Optimize the multi-register read in i915_get_vblank_counter() a little
bit by grabbing the uncore lock manually and using I915_READ_FW().
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170309154434.29303-2-ville.syrjala@linux.intel.comReviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>

gen6_sanitize_rps_pm_mask() is small enough that inlining it shrinks the
object code.

v2: Use const markup
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170312135426.2216-1-chris@chris-wilson.co.ukReviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>

The REDIRECT_TO_GUC bit is a strange beast as it is a disable bit -
setting the bit in the pm interrupt generation stops the interrupt going
to the guc (not sending it to the guc as the name implies). To help the
reader rename it to DISABLE_REDIRECT_TO_GUC so that we keep the bspec
greppable name without it being as confusing!
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Oscar Mateo <oscar.mateo@intel.com>
Cc: Radoslaw Szwichtenberg <radoslaw.szwichtenberg@intel.com>
Cc: Sagar Arun Kamble <sagar.a.kamble@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170312132745.9618-1-chris@chris-wilson.co.ukReviewed-by: NSagar Arun Kamble <sagar.a.kamble@intel.com>

Different state is to be maintained for rps.pm_intrmsk_mbz for GuC and
Execlists. Updating it inside guc_interrupts_* routines as in those
routines GuC load/submission params are sanitized and it should not be set
based on HAS_GUC_SCHED during intel_irq_init.

Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Oscar Mateo <oscar.mateo@intel.com>
Cc: Radoslaw Szwichtenberg <radoslaw.szwichtenberg@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NSagar Arun Kamble <sagar.a.kamble@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1489199821-6707-3-git-send-email-sagar.a.kamble@intel.comSigned-off-by: NChris Wilson <chris@chris-wilson.co.uk>

"pm_intr_keep" is not conveying the intent that it is bitmask
of interrupts that must be zero(mbz) in GEN6_PMINTRMSK.
Name it "pm_intrmsk_mbz".

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Radoslaw Szwichtenberg <radoslaw.szwichtenberg@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NSagar Arun Kamble <sagar.a.kamble@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1489199821-6707-2-git-send-email-sagar.a.kamble@intel.comSigned-off-by: NChris Wilson <chris@chris-wilson.co.uk>

To make our adjustments to RPS requires taking a mutex and potentially
sleeping for an unknown duration - until we have completed our
adjustments further RPS interrupts are immaterial (they are based on
stale thresholds) and we can safely ignore them.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170309211232.28878-3-chris@chris-wilson.co.ukReviewed-by: NMika Kuoppala <mika.kuoppala@linux.intel.com>

Currently, we sum the render and media cycles (on different engines) to
compute a percentage - but we fail to factor in the duplication into the
threshold calculations. This makes us very eager to upclock!

If we just consider the maximum busy cycles of either counter, we should
have an accurate reflection on whether there are cycles to spare to
handle the workload at this frequency.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NMika Kuoppala <mika.kuoppala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170309211232.28878-2-chris@chris-wilson.co.uk

On Baytrail, we manually calculate busyness over the evaluation interval
to avoid issues with miscaluations with RC6 enabled. However, it turns
out that the DOWN_EI interrupt generator is completely bust - it
operates in two modes, continuous or never. Neither of which are
conducive to good behaviour. Stop unmask the DOWN_EI interrupt and just
compute everything from the UP_EI which does seem to correspond to the
desired interval.

v2: Fixup gen6_rps_pm_mask() as well
v3: Inline vlv_c0_above() to combine the now identical elapsed
calculation for up/down and simplify the threshold testing

Fixes: 43cf3bf0 ("drm/i915: Improved w/a for rps on Baytrail")
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: <stable@vger.kernel.org> # v4.1+
Link: http://patchwork.freedesktop.org/patch/msgid/20170309211232.28878-1-chris@chris-wilson.co.ukReviewed-by: NMika Kuoppala <mika.kuoppala@linux.intel.com>

Driver needs to ensure that it doesn't mask the PM interrupts, which are
unmasked/needed by GuC firmware. For that, Driver maintains a bitmask of
interrupts to be kept unmasked, pm_intr_keep.

pm_intr_keep was determined across GuC load. GuC gets loaded in different
scenarios and it is not going to change the pm_intr_keep so this patch
moves its setup to intel_irq_init.

This patch fixes incorrect RPS masking leading to UP interrupts triggered
even when at cur_freq=max and inversly for Down interrupts.

Cc: Radoslaw Szwichtenberg <radoslaw.szwichtenberg@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Michal Winiarski <michal.winiarski@intel.com>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Signed-off-by: NSagar Arun Kamble <sagar.a.kamble@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1488862355-9768-1-git-send-email-sagar.a.kamble@intel.com

As we now take the breadcrumbs spinlock within the interrupt handler, we
wish to minimise its hold time. During the interrupt we do not care
about the state of the full rbtree, only that of the first element, so
we can guard that with a separate lock.

v2: Rename first_wait to irq_wait to make it clearer that it is guarded
by irq_lock.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170303190824.1330-1-chris@chris-wilson.co.uk

Being inside a spinlock signaling that the hardware just completed a
request doesn't prevent a second thread already spotting that the
request is complete, freeing it and reallocating it! The code currently
tries to prevent this using RCU -- but that only prevents the request
from being freed, it doesn't prevent us from reallocating it - that
requires us to take a reference.

[  206.922985] BUG: spinlock already unlocked on CPU#4, gem_exec_parall/7796
[  206.922994]  lock: 0xffff8801c6047120, .magic: dead4ead, .owner: <none>/-1, .owner_cpu: -1
[  206.923000] CPU: 4 PID: 7796 Comm: gem_exec_parall Not tainted 4.10.0-CI-Patchwork_4008+ #1
[  206.923006] Hardware name: System manufacturer System Product Name/Z170M-PLUS, BIOS 1805 06/20/2016
[  206.923012] Call Trace:
[  206.923014]  <IRQ>
[  206.923019]  dump_stack+0x67/0x92
[  206.923023]  spin_dump+0x73/0xc0
[  206.923027]  do_raw_spin_unlock+0x79/0xb0
[  206.923031]  _raw_spin_unlock_irqrestore+0x27/0x60
[  206.923042]  dma_fence_signal+0x160/0x230
[  206.923060]  notify_ring+0xae/0x2e0 [i915]
[  206.923073]  ? ibx_hpd_irq_handler+0xc0/0xc0 [i915]
[  206.923086]  gen8_gt_irq_handler+0x219/0x290 [i915]
[  206.923100]  gen8_irq_handler+0x8e/0x6b0 [i915]
[  206.923105]  __handle_irq_event_percpu+0x58/0x370
[  206.923109]  handle_irq_event_percpu+0x1e/0x50
[  206.923113]  handle_irq_event+0x34/0x60
[  206.923117]  handle_edge_irq+0xbe/0x150
[  206.923122]  handle_irq+0x15/0x20
[  206.923126]  do_IRQ+0x63/0x130
[  206.923142]  ? i915_mutex_lock_interruptible+0x39/0x140 [i915]
[  206.923148]  common_interrupt+0x90/0x90
[  206.923153] RIP: 0010:osq_lock+0x77/0x110
[  206.923157] RSP: 0018:ffffc90001cabaa0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff6e
[  206.923164] RAX: 0000000000000000 RBX: ffff880236d1abc0 RCX: ffff8801ef642fc0
[  206.923169] RDX: ffff8801ef6427c0 RSI: ffffffff81c6e7fd RDI: ffffffff81c7c848
[  206.923175] RBP: ffffc90001cabab8 R08: 00000000692bb19b R09: 08c1493200000000
[  206.923180] R10: 0000000000000001 R11: 0000000000000001 R12: ffff880236cdabc0
[  206.923185] R13: ffff8802207f00b0 R14: ffffffffa00b7cd9 R15: ffff8802207f0070
[  206.923191]  </IRQ>
[  206.923206]  ? i915_mutex_lock_interruptible+0x39/0x140 [i915]
[  206.923213]  __mutex_lock+0x649/0x990
[  206.923217]  ? __mutex_lock+0xb0/0x990
[  206.923221]  ? _raw_spin_unlock+0x2c/0x50
[  206.923226]  ? __pm_runtime_resume+0x56/0x80
[  206.923242]  ? i915_mutex_lock_interruptible+0x39/0x140 [i915]
[  206.923249]  mutex_lock_interruptible_nested+0x16/0x20
[  206.923264]  i915_mutex_lock_interruptible+0x39/0x140 [i915]
[  206.923270]  ? __pm_runtime_resume+0x56/0x80
[  206.923285]  i915_gem_do_execbuffer.isra.15+0x442/0x1d10 [i915]
[  206.923291]  ? __lock_acquire+0x449/0x1b50
[  206.923296]  ? __might_fault+0x3e/0x90
[  206.923301]  ? __might_fault+0x87/0x90
[  206.923305]  ? __might_fault+0x3e/0x90
[  206.923320]  i915_gem_execbuffer2+0xb5/0x220 [i915]
[  206.923327]  drm_ioctl+0x200/0x450
[  206.923341]  ? i915_gem_execbuffer+0x330/0x330 [i915]
[  206.923348]  do_vfs_ioctl+0x90/0x6e0
[  206.923352]  ? __fget+0x108/0x200
[  206.923356]  ? expand_files+0x2b0/0x2b0
[  206.923361]  SyS_ioctl+0x3c/0x70
[  206.923365]  entry_SYSCALL_64_fastpath+0x1c/0xb1
[  206.923369] RIP: 0033:0x7fdd75fc6357
[  206.923373] RSP: 002b:00007fdd20e59bf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[  206.923380] RAX: ffffffffffffffda RBX: ffffffff81481ff3 RCX: 00007fdd75fc6357
[  206.923385] RDX: 00007fdd20e59c70 RSI: 0000000040406469 RDI: 0000000000000003
[  206.923390] RBP: ffffc90001cabf88 R08: 0000000000000040 R09: 00000000000003f7
[  206.923396] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[  206.923401] R13: 0000000000000003 R14: 0000000040406469 R15: 0000000001cf9cb0
[  206.923408]  ? __this_cpu_preempt_check+0x13/0x20

Fixes: 56299fb7 ("drm/i915: Signal first fence from irq handler if complete")
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=100051Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170303144557.4815-1-chris@chris-wilson.co.ukReviewed-by: NMika Kuoppala <mika.kuoppala@linux.intel.com>

Add tracepoints for plane programming. The tracepoints will dump
the frame and scanline counters, so this can be used to verify eg. that
the plane gets reprogrammed at the right time with respect to watermark
programming (if we have appropriate tracepoints for that as well).

v2: Rebase due to legacy cursor changes
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170302171508.1666-16-ville.syrjala@linux.intel.com

assert_spin_locked() becomes an unconditionally compiled BUG_ON(),
adding debug code right into the heart of critical routines like
interrupt handlers.

   text	   data	    bss	    dec	    hex
1296480	  19944	   2272	1318696	 141f28	before (lockdep disabled)
1295984	  19944	   2272	1318200	 141d38	after

1336261	  21139	   3208	1360608	 14c2e0	before (lockdep enabled)
1339920	  21139	   3208	1364267	 14d12b	after

Small saving for release; hopefully more instructive in debug.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170302132801.599-1-chris@chris-wilson.co.ukReviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>

A significant cost in setting up a wait is the overhead of enabling the
interrupt. As we disable the interrupt whenever the queue of waiters is
empty, if we are frequently waiting on alternating batches, we end up
re-enabling the interrupt on a frequent basis. We do want to disable the
interrupt during normal operations as under high load it may add several
thousand interrupts/s - we have been known in the past to occupy whole
cores with our interrupt handler after accidentally leaving user
interrupts enabled. As a compromise, leave the interrupt enabled until
the next IRQ, or the system is idle. This gives a small window for a
waiter to keep the interrupt active and not be delayed by having to
re-enable the interrupt.

v2: Restore hangcheck/missed-irq detection for continuations
v3: Be more careful restoring the hangcheck timer after reset
v4: Be more careful restoring the fake irq after reset (if required!)
v5: Redo changes to intel_engine_wakeup()
v6: Factor out __intel_engine_wakeup()
v7: Improve commentary for declaring a missed wakeup
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170227205850.2828-4-chris@chris-wilson.co.uk

As execlists and other non-semaphore multi-engine devices coordinate
between engines using interrupts, we can shave off a few 10s of
microsecond of scheduling latency by doing the fence signaling from the
interrupt as opposed to a RT kthread. (Realistically the delay adds
about 1% to an individual cross-engine workload.) We only signal the
first fence in order to limit the amount of work we move into the
interrupt handler. We also have to remember that our breadcrumbs may be
unordered with respect to the interrupt and so we still require the
waiter process to perform some heavyweight coherency fixups, as well as
traversing the tree of waiters.

v2: No need for early exit in irq handler - it breaks the flow between
patches and prevents the tracepoint
v3: Restore rcu hold across irq signaling of request
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170227205850.2828-2-chris@chris-wilson.co.uk

i915_gem_ring_notify is more appropriate since we do not have
the request information at this point, but it is simply a
signal from the engine that some request has been completed.

v2:
  * Always trace and log if there were any waiters.
  * Rename to intel_engine_notify. (Chris Wilson)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

When the timer expires for checking on interrupt processing, check to
see if any interrupts arrived within the last time period. If real
interrupts are still being delivered, we can be reassured that we
haven't missed the final interrupt as the waiter will still be woken.
Only once all activity ceases, do we have to worry about the waiter
never being woken and so need to install a timer to kick the waiter for
a slow arrival of a seqno.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170217151304.16665-2-chris@chris-wilson.co.uk

For LSPCON resume time initialization we need to sample the
corresponding pin's HPD level, but this is only available when HPD
detection is enabled. Currently we enable detection only when enabling
HPD interrupts which is too late, so bring the enabling of detection
earlier.

This is needed by the next patch.

Cc: Shashank Sharma <shashank.sharma@intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: <stable@vger.kernel.org> # v4.9+
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NShashank Sharma <shashank.sharma@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1485509961-9010-2-git-send-email-imre.deak@intel.com
(cherry picked from commit 7fff8126)
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

In order to prevent accessing the hpd registers outside of the display
power wells, we should refrain from writing to the registers before the
display interrupts are enabled.

[    4.740136] WARNING: CPU: 1 PID: 221 at drivers/gpu/drm/i915/intel_uncore.c:795 __unclaimed_reg_debug+0x44/0x50 [i915]
[    4.740155] Unclaimed read from register 0x1e1110
[    4.740168] Modules linked in: i915(+) intel_gtt drm_kms_helper prime_numbers
[    4.740190] CPU: 1 PID: 221 Comm: systemd-udevd Not tainted 4.10.0-rc6+ #384
[    4.740203] Hardware name:                  /        , BIOS PYBSWCEL.86A.0027.2015.0507.1758 05/07/2015
[    4.740220] Call Trace:
[    4.740236]  dump_stack+0x4d/0x6f
[    4.740251]  __warn+0xc1/0xe0
[    4.740265]  warn_slowpath_fmt+0x4a/0x50
[    4.740281]  ? insert_work+0x77/0xc0
[    4.740355]  ? fwtable_write32+0x90/0x130 [i915]
[    4.740431]  __unclaimed_reg_debug+0x44/0x50 [i915]
[    4.740507]  fwtable_read32+0xd8/0x130 [i915]
[    4.740575]  i915_hpd_irq_setup+0xa5/0x100 [i915]
[    4.740649]  intel_hpd_init+0x68/0x80 [i915]
[    4.740716]  i915_driver_load+0xe19/0x1380 [i915]
[    4.740784]  i915_pci_probe+0x32/0x90 [i915]
[    4.740799]  pci_device_probe+0x8b/0xf0
[    4.740815]  driver_probe_device+0x2b6/0x450
[    4.740828]  __driver_attach+0xda/0xe0
[    4.740841]  ? driver_probe_device+0x450/0x450
[    4.740853]  bus_for_each_dev+0x5b/0x90
[    4.740865]  driver_attach+0x19/0x20
[    4.740878]  bus_add_driver+0x166/0x260
[    4.740892]  driver_register+0x5b/0xd0
[    4.740906]  ? 0xffffffffa0166000
[    4.740920]  __pci_register_driver+0x47/0x50
[    4.740985]  i915_init+0x5c/0x5e [i915]
[    4.740999]  do_one_initcall+0x3e/0x160
[    4.741015]  ? __vunmap+0x7c/0xc0
[    4.741029]  ? kmem_cache_alloc+0xcf/0x120
[    4.741045]  do_init_module+0x55/0x1c4
[    4.741060]  load_module+0x1f3f/0x25b0
[    4.741073]  ? __symbol_put+0x40/0x40
[    4.741086]  ? kernel_read_file+0x100/0x190
[    4.741100]  SYSC_finit_module+0xbc/0xf0
[    4.741112]  SyS_finit_module+0x9/0x10
[    4.741125]  entry_SYSCALL_64_fastpath+0x17/0x98
[    4.741135] RIP: 0033:0x7f8559a140f9
[    4.741145] RSP: 002b:00007fff7509a3e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[    4.741161] RAX: ffffffffffffffda RBX: 00007f855aba02d1 RCX: 00007f8559a140f9
[    4.741172] RDX: 0000000000000000 RSI: 000055b6db0914f0 RDI: 0000000000000011
[    4.741183] RBP: 0000000000020000 R08: 0000000000000000 R09: 000000000000000e
[    4.741193] R10: 0000000000000011 R11: 0000000000000246 R12: 000055b6db0854d0
[    4.741204] R13: 000055b6db091150 R14: 0000000000000000 R15: 000055b6db035924

v2: Set dev_priv->display_irqs_enabled to true for all platforms other
than vlv/chv that manually control the display power domain.

Fixes: 19625e85 ("drm/i915: Enable polling when we don't have hpd")
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=97798Suggested-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Lyude <cpaul@redhat.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Hans de Goede <jwrdegoede@fedoraproject.org>
Cc: stable@vger.kernel.org
Link: http://patchwork.freedesktop.org/patch/msgid/20170215131547.5064-1-chris@chris-wilson.co.ukReviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>

Currently we apply the jump to rpe if we are below it and the GPU needs
more power. For some GPUs, the rpe is 75% of the maximum range causing
us to dramatically overshoot low power applications *and* unable to
reach the low frequency that can most efficiently deliver their
workload.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/20170210150348.22146-3-chris@chris-wilson.co.ukReviewed-by: NRadoslaw Szwichtenberg <radoslaw.szwichtenberg@intel.com>

If we receive a DOWN_TIMEOUT rps interrupt, we respond by reducing the
GPU clocks significantly. Before we do, double check that the frequency
we pick is actually a decrease.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/20170210150348.22146-2-chris@chris-wilson.co.ukReviewed-by: NRadoslaw Szwichtenberg <radoslaw.szwichtenberg@intel.com>

This adds a file in i915's debugfs directory that allows userspace to
manually control HPD storm detection. This is mainly for hotplugging
tests, where we might want to test HPD storm functionality or disable
storm detection to speed up hotplugging tests without breaking anything.

Changes since v1:
- Make HPD storm interval configurable
- Misc code cleanup
Signed-off-by: NLyude <lyude@redhat.com>
Acked-by: NJani Nikula <jani.nikula@linux.intel.com>
Cc: Tomeu Vizoso <tomeu@tomeuvizoso.net>

Core code already makes drm_driver.get_vblank_counter hook optional by
letting drm_vblank_no_hw_counter be the default implementation for the
function hook.  So the drm_vblank_no_hw_counter assignment in the driver
code becomes redundant and can be removed now.
Signed-off-by: NShawn Guo <shawn.guo@linaro.org>
Cc: Alexey Brodkin <abrodkin@synopsys.com>
Cc: Liviu Dudau <liviu.dudau@arm.com>
Cc: Mali DP Maintainers <malidp@foss.arm.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Boris Brezillon <boris.brezillon@free-electrons.com>
Cc: Inki Dae <inki.dae@samsung.com>
Cc: Stefan Agner <stefan@agner.ch>
Cc: Xinliang Liu <z.liuxinliang@hisilicon.com>
Cc: Daniel Vetter <daniel.vetter@intel.com>
Cc: Philipp Zabel <p.zabel@pengutronix.de>
Cc: CK Hu <ck.hu@mediatek.com>
Cc: Neil Armstrong <narmstrong@baylibre.com>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Marek Vasut <marex@denx.de>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Tomi Valkeinen <tomi.valkeinen@ti.com>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Mark Yao <mark.yao@rock-chips.com>
Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
Cc: Maxime Ripard <maxime.ripard@free-electrons.com>
Cc: Jyri Sarha <jsarha@ti.com>
Cc: Eric Anholt <eric@anholt.net>
Acked-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NBoris Brezillon <boris.brezillon@free-electrons.com>
Acked-by: NNeil Armstrong <narmstrong@baylibre.com>
Acked-by: NRussell King <rmk+kernel@armlinux.org.uk>
Reviewed-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: NAlexey Brodkin <abrodkin@synopsys.com>
Acked-by: NLiviu Dudau <liviu.dudau@arm.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1486458995-31018-3-git-send-email-shawnguo@kernel.org

To be consistent with the recent change to enable hotplug detection
early on GEN9 platforms do the same on all non-GMCH platforms starting
from GEN5. On GMCH platforms enabling detection without interrupts isn't
trivial, since AUX and HPD have a shared interrupt line. It could be
done there too by using a SW interrupt mask, but I punt on that for now.

Cc: Shashank Sharma <shashank.sharma@intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NShashank Sharma <shashank.sharma@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1485509961-9010-5-git-send-email-imre.deak@intel.com