This allows writes to EU flow control registers. Together
with SIP code from the user-mode driver this resolves a
hang seen in some pre-emption scenarios. Note that this
patch is just the kernel mode part of this workaround.

v2. Oops, add FLOW_CONTROL_ENABLE macro to i915_reg.h.
Signed-off-by: NTim Gore <tim.gore@intel.com>
Reviewed-by: NArun Siluvery <arun.siluvery@linux.intel.com>
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1458144826-17269-1-git-send-email-tim.gore@intel.com

By reading the CSB (slow MMIO accesses) into a temporary local
buffer we can decrease the duration of holding the execlist
lock.

Main advantage is that during heavy batch buffer submission we
reduce the execlist lock contention, which should decrease the
latency and CPU usage between the submitting userspace process
and interrupt handling.

Downside is that we need to grab and relase the forcewake twice,
but as the below numbers will show this is completely hidden
by the primary gains.

Testing with "gem_latency -n 100" (submit batch buffers with a
hundred nops each) shows more than doubling of the throughput
and more than halving of the dispatch latency, overall latency
and CPU time spend in the submitting process.

Submitting empty batches ("gem_latency -n 0") does not seem
significantly affected by this change with throughput and CPU
time improving by half a percent, and overall latency worsening
by the same amount.

Above tests were done in a hundred runs on a big core Broadwell.

v2:
  * Overflow protection to local CSB buffer.
  * Use closer dev_priv in execlists_submit_requests. (Chris Wilson)

v3: Rebase.

v4: Added commend about irq needed to be disabled in
    execlists_submit_request. (Chris Wilson)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NChris Wilsno <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458219586-20452-1-git-send-email-tvrtko.ursulin@linux.intel.com

Where we have a request we can use req->i915 directly instead
of going through the engine and device. Coccinelle script:

@@
function f;
identifier r;
@@
f(..., struct drm_i915_gem_request *r, ...)
{
...
- engine->dev->dev_private
+ r->i915
...
}
@@
struct drm_i915_gem_request *req;
@@
(
  req->
- engine->dev->dev_private
+ i915
)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458219850-21007-1-git-send-email-tvrtko.ursulin@linux.intel.com

Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Add support for forcing an error at selected places in the driver. As an
example add 4 options to fail during driver loading.

Requested by Chris.

v2:
- Add fault point for modeset initialization
- Print debug message when injecting an error
v3:
- Rename inject_fault to inject_load_failure, rename the related macros
  and helper accordingly (Chris)
- Use a counter instead of a mask to identify the failure point (Daniel)
- Mark the module option as _unsafe and keep i915_params ordered (Joonas)
v4:
- Rebase on latest -nightly
v5:
- Use DRM_INFO instead of DRM_DEBUG_DRIVER, making it clearer in CI reports
  that a following error message is expected (IRC r-b from Chris on v5)

CC: Chris Wilson <chris@chris-wilson.co.uk>
CC: Daniel Vetter <daniel.vetter@ffwll.ch>
CC: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Move the cleanup of the power domain HW state on the error path to the
same function where the corresponding init call was called from. I
noticed this problem when loading the module with load failure injection
enabled, making i915_load_modeset_init() fail.

CC: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-19-git-send-email-imre.deak@intel.com

According to the new init phases scheme we should register the device
making it available via some kernel internal or user space interface as
the last step in the init sequence, so move the corresponding code to a
separate function.

Also add a TODO comment about code that still needs to be moved around
to one of the init phases functions depending on what the role and effect
of that code is.

No functional change, except for the reordering of the unload time
unregistration steps of sysfs wrt. acpi and opregion.

Suggested by Chris.

v3:
- rename i915_driver_init_register to i915_driver_init_frameworks
  (Chris)
- rename i915_driver_init_frameworks to i915_driver_register (Daniel)

CC: Chris Wilson <chris@chris-wilson.co.uk>
CC: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-18-git-send-email-imre.deak@intel.com

According to the new init phases scheme we should have a definite step
in the init sequence where we setup things requiring accessing the
device, so move the corresponding code to separate function. The steps
in this init phase should avoid exposing the driver via some interface,
which is done in the last registration init phase. This changae also
has the benefit of making the error path cleaner both in the new
function and i915_driver_load()/unload().

No functional change.

Suggested by Chris.

CC: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-17-git-send-email-imre.deak@intel.com

According to the new init phases scheme we should have a definite step
in the init sequence where MMIO access is setup, so move the
corresponding code to a separate function. This also has the benefit of
making the error path cleaner both in the new function and in
i915_driver_load()/unload().

No functional change.

Suggested by Chris.

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-16-git-send-email-imre.deak@intel.com

According to the new init phases scheme we should initialize "SW-only"
state not requiring accessing the device as the very first step, so that
the reasoning about dependencies of later steps becomes easier. So move
these init steps into a separate function. This also has the benefit of
making the error path cleaner both in the new function and int
i915_driver_load()/unload().

No functional change.

Suggested by Chris.

CC: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-15-git-send-email-imre.deak@intel.com

Move the opregion unregistration earlier to match its corresponding
registration order.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-14-git-send-email-imre.deak@intel.com

Move the GTT,MSI IRQ cleanup later so that it matches their
corresponding init order. Also fix the order of these calls wrt. each
other to match their corresponding init order.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-13-git-send-email-imre.deak@intel.com

Move the power domain uninitialization later so that it matches its
corresponding init order. Since we access the HW during the later
unitialization steps keep a wake reference until after the last such
step.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-12-git-send-email-imre.deak@intel.com

We should register all the interfaces before we enable runtime PM.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-11-git-send-email-imre.deak@intel.com

According to the new init phases scheme we should register the driver
with frameworks/userspace only one the device is setup fully. So move
the shrinker registration later accordingly.

Also fix the shrinker unregistration order wrt. the acpi unregistration
to fix the corresponding init order.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-10-git-send-email-imre.deak@intel.com

We require the device to be powered only before accessing it, so we can
move this call later.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-9-git-send-email-imre.deak@intel.com

The only steps requiring device access is the fence and swizzling
initialization, so split these out keeping them in their current place
and move the rest of init steps earlier.

v2-v3:
- unchanged
v4:
- move call to i915_gem_detect_bit_6_swizzle() to
  i915_gem_load_init_fences() and preserve the original order of
  the detection of HW fence capailities wrt. swizzling (Chris)

CC: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458132843-21860-1-git-send-email-imre.deak@intel.com

This init step accesses the device, but doesn't have any device
specific side effect. It also sets up some platform specific
attributes that may be required early, so move it earlier.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-7-git-send-email-imre.deak@intel.com

Split out the part initing the clock gating hooks and move it earlier.
Add a new NOP hook for platforms without the need to apply clockgating
or workaround settings, so that the hook can be called unconditionally.
Also add a WARN for future platforms that forget to add a hook.

The rest of the hooks in intel_init_pm() should be inited in the same
way, but atm some of the hooks are set only conditionally, so before
doing this we need to make the setup unconditional and use instead some
flags.

v2:
- add a NOP hook and WARN if no hook is set for the platform (Chris)
- use the term hook instead of callback for these functions (Jani)
v3:
- remove the GEN4() check it's already covered by earlier platform
  checks (Chris)

CC: Jani Nikula <jani.nikula@intel.com>
CC: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-6-git-send-email-imre.deak@intel.com

All of this is SW only initialization so we can move them earlier. Move
the mutex init where the rest of the locks are inited. While at it also
convert dev to dev_priv.

v2:
- use the term hook instead of callback for these functions (Jani)

CC: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-5-git-send-email-imre.deak@intel.com

Most of the IRQ init is setting up hooks so move that part earlier.
Leave the pm_qos_add_request() call in place.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-4-git-send-email-imre.deak@intel.com

These are all SW only init steps not accessing the device and they only
need the platform identification macros to work, which are already
available earlier, so move these init steps earlier.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-3-git-send-email-imre.deak@intel.com

MCHBAR is cleaned up in i915_mmio_cleanup(), so the separate call in
i915_driver_load() is incorrect.

CC: David Weinehall <david.weinehall@intel.com>
Fixes: ad5c3d3f ("drm/i915: Move MCHBAR setup earlier during init")
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NDavid Weinehall <david.weinehall@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-2-git-send-email-imre.deak@intel.com

In full gpu reset we prime all engines and reset domains corresponding to
each engine. Per engine reset is just a special case of this process
wherein only a single engine is reset. This change is aimed to modify
relevant functions to achieve this. There are some other steps we carry out
in case of engine reset which are addressed in later patches.

Reset func now accepts a mask of all engines that need to be reset. Where
per engine resets are supported, error handler populates the mask
accordingly otherwise all engines are specified.

v2: ALL_ENGINES mask fixup, better for_each_ring_masked (Chris)
v3: Whitespace fixes (Chris)
v4: Rebase due to s/ring/engine

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NArun Siluvery <arun.siluvery@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1458143640-20563-1-git-send-email-mika.kuoppala@intel.com

We've been accumulating code across the driver that depends on the VBT
specific structures and defines. The VBT is an uncontrollable
beast. Encourage encapsulation of the VBT data by hiding the structures
and defines in a private header only to be included from intel_bios.c.
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1458125015-7931-7-git-send-email-jani.nikula@intel.com

drivers/gpu/drm/i915/intel_dpll_mgr.c:1200:32: warning: Using plain integer as NULL pointer

Fixes: 304b65cb ("drm/i915: Move SKL/KLB pll selection logic to intel_dpll_mgr.c")
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1458144418-20046-1-git-send-email-jani.nikula@intel.com

Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1457944075-14123-4-git-send-email-maarten.lankhorst@linux.intel.comReviewed-by: NAnder Conselvan de Oliveira <conselvan2@gmail.com>

Warn for the wrong mask in enable only. Disable will have the wrong mask now
because the new state is committed before disabling the old state.

Changes since v1:
- Use crtc_mask (Durgadoss)
- Rebase.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1457944075-14123-3-git-send-email-maarten.lankhorst@linux.intel.comReviewed-by: NAnder Conselvan de Oliveira <conselvan2@gmail.com>

This makes it easier to verify correct dpll setup with only a single crtc.
It is also useful to detect double dpll enable/disable.

Changes since v1:
- Rebase on top of Ander's dpll rework.
- Change debugfs active to a mask.
- Change enabled_crtcs and active_crtcs to unsigned.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1457944075-14123-2-git-send-email-maarten.lankhorst@linux.intel.comReviewed-by: NAnder Conselvan de Oliveira <conselvan2@gmail.com>

After unplugging a DP MST display from the system, we have to go through
and destroy all of the DRM connectors associated with it since none of
them are valid anymore. Unfortunately, intel_dp_destroy_mst_connector()
doesn't do a good enough job of ensuring that throughout the destruction
process that no modesettings can be done with the connectors. As it is
right now, intel_dp_destroy_mst_connector() works like this:

* Take all modeset locks
* Clear the configuration of the crtc on the connector, if there is one
* Drop all modeset locks, this is required because of circular
  dependency issues that arise with trying to remove the connector from
  sysfs with modeset locks held
* Unregister the connector
* Take all modeset locks, again
* Do the rest of the required cleaning for destroying the connector
* Finally drop all modeset locks for good

This only works sometimes. During the destruction process, it's very
possible that a userspace application will attempt to do a modesetting
using the connector. When we drop the modeset locks, an ioctl handler
such as drm_mode_setcrtc has the oppurtunity to take all of the modeset
locks from us. When this happens, one thing leads to another and
eventually we end up committing a mode with the non-existent connector:

	[drm:intel_dp_link_training_clock_recovery [i915]] *ERROR* failed to enable link training
	[drm:intel_dp_aux_ch] dp_aux_ch timeout status 0x7cf0001f
	[drm:intel_dp_start_link_train [i915]] *ERROR* failed to start channel equalization
	[drm:intel_dp_aux_ch] dp_aux_ch timeout status 0x7cf0001f
	[drm:intel_mst_pre_enable_dp [i915]] *ERROR* failed to allocate vcpi

And in some cases, such as with the T460s using an MST dock, this
results in breaking modesetting and/or panicking the system.

To work around this, we now unregister the connector at the very
beginning of intel_dp_destroy_mst_connector(), grab all the modesetting
locks, and then hold them until we finish the rest of the function.

CC: stable@vger.kernel.org
Signed-off-by: NLyude <cpaul@redhat.com>
Signed-off-by: NRob Clark <rclark@redhat.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1458155884-13877-1-git-send-email-cpaul@redhat.com

This version of GuC firmware fixes the engine reset issue where golden
context LRC address is treated as page index by mistake. It also fixes
the problem that scheduler stops submiting to one engine when the other
engine work queue is full.
Signed-off-by: NAlex Dai <yu.dai@intel.com>
Reviewed-by: NDave Gordon <david.s.gordon@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>

Favor a single point of truth instead of duplicating the
information. The change also filters out unsupported DSI ports at this
stage, accepting only ports A and C, instead of waiting until the port
checks.
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1458125015-7931-6-git-send-email-jani.nikula@intel.com

Use the connector type instead of VBT directly to decide which backlight
mechanism to use on VLV/CHV. (Indirectly, this is the same thing, but
hides the VBT use.)
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1458125015-7931-5-git-send-email-jani.nikula@intel.com

Hide knowledge about VBT child devices in intel_bios.c.

v2: Move port check to intel_bios.c (Sivakumar)
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1458125015-7931-4-git-send-email-jani.nikula@intel.com

Hide knowledge about VBT child devices in intel_bios.c.
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1458125015-7931-3-git-send-email-jani.nikula@intel.com

Hide knowledge about VBT child devices in intel_bios.c.
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1458125015-7931-2-git-send-email-jani.nikula@intel.com

Hide knowledge about VBT child devices in intel_bios.c.

v2: also move int_tv_support check to intel_bios.c (Sivakumar)
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1458125015-7931-1-git-send-email-jani.nikula@intel.com

BXT isn't as limited as BYT and CHT regarding DSI pipes and ports.
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/8bea85b86aaf7a15f854a656bf8d3208a3afe0bd.1458070700.git.jani.nikula@intel.com

Make the code easier to read and update. No functional changes.
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/04b20a9be6a9481ca2eb2a42cdcdc27358152e5f.1458070700.git.jani.nikula@intel.com

Nicer for eDP (actually "EDP" instead of "D"), and makes future
expansion for DSI transcoders easier.
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/2ac55ed584e450fe154daecb6453dff8eede5e7d.1458070700.git.jani.nikula@intel.com