These helpers will be needed by the next patch, so factor them out.

No functional change.

v2:
- Move the refcount==0 WARN to the new put helper. (Ville)

CC: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>

A follow-up patch moves the PHY enabling to the power well code where
enabling/disabling the PHYs will happen independently. Because of this
waiting for the GRC calibration in PHY1 asynchronously would need some
additional logic. Instead of adding that let's keep things simple for now
and wait synchronously. My measurements showed that the calibration
takes ~4ms.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>

Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465821536-21312-1-git-send-email-kraxel@redhat.com

It seems pretty clear that bitwise OR was intended here and not logical
OR.

Fixes: 6fc29133 ('drm/i915/gen9: Add WaDisableSkipCaching')
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>

Calling drm_framebuffer_unregister_private() in intel_fbdev_destroy() is
superfluous because the framebuffer will subsequently be unregistered by
drm_framebuffer_free() when unreferenced in drm_framebuffer_remove().
The call is a leftover, when it was introduced by commit 36206361
("drm: revamp framebuffer cleanup interfaces"), struct intel_framebuffer
was still embedded in struct intel_fbdev rather than being a pointer as
it is today, and drm_framebuffer_remove() wasn't used yet.

As a bonus, the ID of the framebuffer is no longer 0 in the debug log:

Before:
    [   39.680874] [drm:drm_mode_object_unreference] OBJ ID: 0 (3)
    [   39.680878] [drm:drm_mode_object_unreference] OBJ ID: 0 (2)
    [   39.680884] [drm:drm_mode_object_unreference] OBJ ID: 0 (1)

After:
    [  102.504649] [drm:drm_mode_object_unreference] OBJ ID: 45 (3)
    [  102.504651] [drm:drm_mode_object_unreference] OBJ ID: 45 (2)
    [  102.504654] [drm:drm_mode_object_unreference] OBJ ID: 45 (1)
Signed-off-by: NLukas Wunner <lukas@wunner.de>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/5031860caad67faa0f1be5965331ef048a311a01.1465383212.git.lukas@wunner.de

This patch enables a workaround for a mid thread preemption
issue where a hardware timing problem can prevent the
context restore from happening, leading to a hang.

v2: move to gen9_init_workarounds (Arun)
v3: move to start of gen9_init_workarounds (Arun)
Signed-off-by: NTim Gore <tim.gore@intel.com>
Reviewed-by: NArun Siluvery <arun.siluvery@linux.intel.com>
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465816501-25557-1-git-send-email-tim.gore@intel.com

This patch adds support for extending the pread/pwrite functionality
for objects not backed by shmem. The access will be made through
gtt interface. This will cover objects backed by stolen memory as well
as other non-shmem backed objects.

v2: Drop locks around slow_user_access, prefault the pages before
access (Chris)

v3: Rebased to the latest drm-intel-nightly (Ankit)

v4: Moved page base & offset calculations outside the copy loop,
corrected data types for size and offset variables, corrected if-else
braces format (Tvrtko/kerneldocs)

v5: Enabled pread/pwrite for all non-shmem backed objects including
without tiling restrictions (Ankit)

v6: Using pwrite_fast for non-shmem backed objects as well (Chris)

v7: Updated commit message, Renamed i915_gem_gtt_read to i915_gem_gtt_copy,
added pwrite slow path for non-shmem backed objects (Chris/Tvrtko)

v8: Updated v7 commit message, mutex unlock around pwrite slow path for
non-shmem backed objects (Tvrtko)

v9: Corrected check during pread_ioctl, to avoid shmem_pread being
called for non-shmem backed objects (Tvrtko)

v10: Moved the write_domain check to needs_clflush and tiling mode check
to pwrite_fast (Chris)

v11: Use pwrite_fast fallback for all objects (shmem and non-shmem backed),
call fast_user_write regardless of pagefault in previous iteration

v12: Use page-by-page copy for slow user access too (Chris)

v13: Handled EFAULT, Avoid use of WARN_ON, put_fence only if whole obj
pinned (Chris)

v14: Corrected datatypes/initializations (Tvrtko)

Testcase: igt/gem_stolen, igt/gem_pread, igt/gem_pwrite
Signed-off-by: NAnkitprasad Sharma <ankitprasad.r.sharma@intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465548783-19712-1-git-send-email-ankitprasad.r.sharma@intel.com

In pwrite_fast, map an object page by page if obj_ggtt_pin fails. First,
we try a nonblocking pin for the whole object (since that is fastest if
reused), then failing that we try to grab one page in the mappable
aperture. It also allows us to handle objects larger than the mappable
aperture (e.g. if we need to pwrite with vGPU restricting the aperture
to a measely 8MiB or something like that).

v2: Pin pages before starting pwrite, Combined duplicate loops (Chris)

v3: Combined loops based on local patch by Chris (Chris)

v4: Added i915 wrapper function for drm_mm_insert_node_in_range (Chris)

v5: Renamed wrapper function for drm_mm_insert_node_in_range (Chris)

v5: Added wrapper for drm_mm_remove_node() (Chris)

v6: Added get_pages call before pinning the pages (Tvrtko)
Added remove_mappable_node() wrapper for drm_mm_remove_node() (Chris)

v7: Added size argument for insert_mappable_node (Tvrtko)

v8: Do not put_pages after pwrite, do memset of node in the wrapper
function (insert_mappable_node) (Chris)

v9: Rebase (Ankit)
Signed-off-by: NAnkitprasad Sharma <ankitprasad.r.sharma@intel.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>

This utility function is a companion to i915_gem_object_get_page() that
uses the same cached iterator for the scatterlist to perform fast
sequential lookup of the dma address associated with any page within the
object.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NAnkitprasad Sharma <ankitprasad.r.sharma@intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>

Introduced a new vm specfic callback insert_page() to program a single pte in
ggtt or ppgtt. This allows us to map a single page in to the mappable aperture
space. This can be iterated over to access the whole object by using space as
meagre as page size.

v2: Added low level rpm assertions to insert_page routines (Chris)

v3: Added POSTING_READ post register write (Tvrtko)

v4: Rebase (Ankit)

v5: Removed wmb() and FLUSH_CTL from insert_page, caller to take care
of it (Chris)

v6: insert_page not working correctly without FLSH_CNTL write, added the
write again.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NAnkitprasad Sharma <ankitprasad.r.sharma@intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>

If the user doesn't override the default values of the GuC-related
kernel parameters, then on a non-GuC-based platform we shouldn't
mention that we haven't loaded the GuC firmware.

The various messages have been reordered into a least->most severe
cascade (none/INFO/INFO/ERROR) for ease of comprehension.
Signed-off-by: NDave Gordon <david.s.gordon@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465575685-34169-1-git-send-email-david.s.gordon@intel.com

This reverts commit f165d283.

It breaks one of our CI systems. Quoting from Ville:

[   13.100979] [drm:ironlake_init_pch_refclk] has_panel 1 has_lvds 1 has_ck505 0 using_ssc_source 1
[   13.101413] ------------[ cut here ]------------
[   13.101429] kernel BUG at drivers/gpu/drm/i915/intel_display.c:8528!

"which is the 'BUG_ON(val != final)' at the end of ironlake_init_pch_refclk()."

Cc: stable@vger.kernel.org
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Lyude <cpaul@redhat.com>
Cc: marius.c.vlad@intel.com
References: https://www.spinics.net/lists/dri-devel/msg109557.htmlAcked-by: NLyude <cpaul@redhat.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>

Add this fbc related workaround for all gen9

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-28-git-send-email-mika.kuoppala@intel.com

Bspec states that we need to set nuke on modify all to prevent
screen corruption with fbc on skl and kbl.

v2: proper workaround name

References: HSD#2227109, HSDES#1404569388
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-27-git-send-email-mika.kuoppala@intel.com

Set bit 8 in 0x43224 to prevent screen corruption and system
hangs on high memory bandwidth conditions. The same wa also suggest
setting bit 31 on ARB_CTL. According to another workaround we gain
better idle power savings when FBC is enabled.

v2: use correct workaround name
v3: split out overlapping wa for corruption avoidance (Ville)

References: HSD#2137218, HSD#2227171, HSD#2136579, BSID#883
Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-26-git-send-email-mika.kuoppala@intel.com

According to bspec this prevents screen corruption when fbc is
used.

v2: This workaround has a name, use it (Ville)
v3: remove bogus gen check on ilk/vlv wm path (Ville)

References: HSD#2135555, HSD#2137270, BSID#562
Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-25-git-send-email-mika.kuoppala@intel.com

This workaround for bdw and chv, is also needed for kbl A0.

References: HSD#1911519, BSID#569
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-24-git-send-email-mika.kuoppala@intel.com

There is ambiguity in the documentation between D0 and E0.
Extend this workaround to E0.

References: BSID#779
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-23-git-send-email-mika.kuoppala@intel.com

Workaround for display underrun issues with Y & Yf Tiling.
Set this on all gen9 as stated by bspec.

v2: proper workaround name

References: HSD#2136383, BSID#857
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-22-git-send-email-mika.kuoppala@intel.com

This is needed for all kbl revision.

v2: Don't add revid checks to generic gen9 init (Arun)

References: HSD#2135593
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-21-git-send-email-mika.kuoppala@intel.com

We need to disable clock gating in this unit to work around
hardware issue causing possible corruption/hang.

v2: name the bit (Ville)
v3: leave the fix enabled for 2227050 and set correct bit (Matthew)
v4: Split out the skl part in separate commit for easier backport

References: HSD#2227156, HSD#2227050
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Matthew Auld <matthew.william.auld@gmail.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-20-git-send-email-mika.kuoppala@intel.com

Add this workaround for A0 and B0 revisions

References: HSD#2226935
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-19-git-send-email-mika.kuoppala@intel.com

This workaround is for fbc working with rc6 on skylake. Bspec
states that setting this bit needs to be coordinated with uncore
but offers no further details.

v2: rebase

References: HSD#4712857
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-18-git-send-email-mika.kuoppala@intel.com

Make sure that we never enable skip caching on gen9 by
accident.

References: HSD#2134698
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-17-git-send-email-mika.kuoppala@intel.com

Add this workaround for both bxt and kbl up to until
rev B0.

References: HSD#2136703
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-16-git-send-email-mika.kuoppala@intel.com

Bspec states that we need to turn off dynamic credit
sharing on kbl revid a0 and b0. This happens by writing bit 28
on 0x4ab8.

References: HSD#2225601, HSD#2226938, HSD#2225763
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-15-git-send-email-mika.kuoppala@intel.com

According to bspec we need to disable gam unit clock gating on
on kbl revids A0 and B0.

References: HSD#2226858, HSD#1944358
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-14-git-send-email-mika.kuoppala@intel.com

The bspec states that these must be set in CONFIG0 for all gen9.

v2: rebase
v3: fix spacing (Matthew)

References: HSD#2134995
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-13-git-send-email-mika.kuoppala@intel.com

Extend the scope of this workaround, already used in skl,
to also take effect in kbl.

v2: Fix KBL_REVID_E0 (Matthew)

References: HSD#2132677
Cc: Matthew Auld <matthew.william.auld@gmail.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-12-git-send-email-mika.kuoppala@intel.com

According to bspec this workaround helps to reduce lag and improve
performance on edp.

Documentation suggests this for bdw and all gen9. However evidence
shows that this register is missing on gen9 and causing unclaimed mmio
access if we access it. So apply to bdw only where the reg
exists and can hold its value.

v2: drop skl

References: HSD#2134579
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-11-git-send-email-mika.kuoppala@intel.com

Add this workaround until upto kbl revid B0.

References: HSD#1802092
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-10-git-send-email-mika.kuoppala@intel.com

Add this workaround for kbl revid A0 only.

v2: rebase
v3: carve out a non related workaround (Chris)

References: HSD#1911714
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-9-git-send-email-mika.kuoppala@intel.com

We need this crucial workaround from skl also to all kbl revisions.
Lack of it was causing system hangs on skl enabling so this is
a must have.

v2: Don't add revid checks to gen9 init workarounds (Arun)

References: HSD#2126660
Cc: Arun Siluvery <arun.siluvery@linux.intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-8-git-send-email-mika.kuoppala@intel.com

Past evidence with system hangs and hsds tie
WaForceEnableNonCoherent and WaDisableHDCInvalidation to
WaForceContextSaveRestoreNonCoherent. Documentation
states that WaForceContextSaveRestoreNonCoherent would
not be needed on skl past E0 but evidence proved otherwise. See
commit <510650e8> ("drm/i915/skl: Fix spurious gpu hang with gt3/gt4
revs"). In this scope consider kbl to be skl with a bigger revision than
E0 so play it safe and bind these two workarounds to the
WaForceContextSaveRestoreNonCoherent, and apply to all gen9.

v2: fix comment (Matthew)

References: HSD#2134449, HSD#2131413
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-7-git-send-email-mika.kuoppala@intel.com

The revision id range for this workaround has changed. So apply
it to all revids on all gen9.

References: HSD#2134449
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-6-git-send-email-mika.kuoppala@intel.com

We need this for kbl a0 boards. Note that this should be also
for bxt A0 but we omit that on purpose as bxt A0's are
out of fashion already.

References: HSD#1912158, HSD#4393097
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-5-git-send-email-mika.kuoppala@intel.com

Add REVID macro for kbl to limit wa applicability to particular
revision range.
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-4-git-send-email-mika.kuoppala@intel.com

Kabylake is part of gen9 family so init the generic gen9
workarounds for it.

v2: rebase
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-3-git-send-email-mika.kuoppala@intel.com

We need to disable clock gating in this unit to work around
hardware issue causing possible corruption/hang.

v2: name the bit (Ville)
v3: leave the fix enabled for 2227050 and set correct bit (Matthew)

References: HSD#2227156, HSD#2227050
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Matthew Auld <matthew.william.auld@gmail.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1465309159-30531-2-git-send-email-mika.kuoppala@intel.com

The atomic version of intel_pre_plane_update did not check
for HAS_GMCH_DISPLAY before calling intel_set_memory_cxsr().
While this doesn't cause any issues on its own (it will
return without doing anything if the hardware doesn't
have the required feature), the drm_wait_one_vblank() that
is needed if memory self-refresh is disabled introduces
an unnecessary delay in the suspend path.

In cases where i915 is on the critical path it means that
we slow down suspend by 16.8ms on platforms that don't
need to disable memory self-refresh.
Signed-off-by: NDavid Weinehall <david.weinehall@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1463662236-18192-1-git-send-email-david.weinehall@linux.intel.comReviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>