Replace the handcrafter loop when checking for fifo slots
with atomic wait for. This brings this wait in line with
the other waits on register access. We also get a readable
timeout constraint, so make it to fail after 10ms.

Chris suggested that we should fail silently as the fifo debug
handler, now attached to unclaimed mmio handling, will take care of the
possible errors at later stage.

Note that the decision to wait was changed so that we avoid
allocating the first reserved entry. Nor do we reduce the count
if we fail the wait, removing the possiblity to wrap the
count if the hw fifo returned zero.

v2: remove unclaimed check on timeout (Chris)
v3: use void return (Chris)

References: https://bugs.freedesktop.org/show_bug.cgi?id=100247Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1491493182-31540-1-git-send-email-mika.kuoppala@intel.com

Remove the per-mmio checking of the FIFO debug register into the common
conditional mmio debug handling. Based on patch from Chris Wilson.

v2: postpone warn on fifodbg for unclaimed reg debugs
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Looks like intel_guc_reset had the ability to sleep under the
uncore spinlock since forever but it wasn't detected until the
recent changes annotated the wait for register with might_sleep.

I have fixed it by removing holding of the uncore spinlock over
the call to gen6_hw_domain_reset, since I do not see that is
really needed. But there is always a possibility I am missing
some nasty detail so please double check.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Oscar Mateo <oscar.mateo@intel.com>
Acked-by: NMichel Thierry <michel.thierry@intel.com>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>

As all other functions related to resetting engines are using
reset_engine.

v2: remove _request_ and use start/cancel instead (Chris)

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NMichel Thierry <michel.thierry@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170418202335.35232-3-michel.thierry@intel.comReviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Provide dummy function pointers for the mock device in case we do hit
mmio during testing.

v2: Use ASSIGN_READ/WRITE_MMIO_FUNCS macros
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170412092143.3822-1-chris@chris-wilson.co.uk

We acquire the forcewake and use I915_READ_FW instead for the atomic
wait within intel_uncore_wait_for_register. However, this still leaves
us vulnerable to concurrent mmio access to the register, which can cause
system hangs on gen7. The protection is to acquire uncore.lock around
each register, so lets add it back.

v2: Wrap __intel_wait_for_register_fw() to re-use its atomic wait_for
loop and spare adding another for ourselves.
v3: Add might_sleep() annotation
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170411101340.31994-3-chris@chris-wilson.co.ukReviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>

Allow the caller to use the fast_timeout_us to specify how long to wait
within the atomic section, rather than transparently switching to a
sleeping loop for larger values. This is required as some callsites may
need a long wait and are in an atomic section.

v2: Reinforce kerneldoc fast_timeout_us limit with a GEM_BUG_ON
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170411112705.12656-1-chris@chris-wilson.co.ukReviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>

This function should not be called with long timeouts in atomic context.
Annotate it as might_sleep if timeout is longer than 10us.

v2: fix comment (Michal)
Signed-off-by: NMichal Wajdeczko <michal.wajdeczko@intel.com>
Suggested-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/20170410121747.209200-1-michal.wajdeczko@intel.comReviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

These params are passed by value, const qualifiers are ignored any way.
While around, unify timeout_ms type from long to int.
Signed-off-by: NMichal Wajdeczko <michal.wajdeczko@intel.com>
Suggested-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/20170410093817.151280-1-michal.wajdeczko@intel.comReviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

In some cases we may want to spend more time in atomic wait than
hardcoded 2us. Let's add additional fast timeout parameter to allow
flexible configuration of atomic timeout before switching into heavy wait.
Add also possibility to return registry value to avoid extra read.

v2: use explicit fast timeout (Tvrtko/Chris)
    allow returning register value (Chris)
Signed-off-by: NMichal Wajdeczko <michal.wajdeczko@intel.com>
Suggested-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/20170407160145.181328-1-michal.wajdeczko@intel.comReviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

There is no need to specify timeout as unsigned long since this parameter
will be consumed by usecs_to_jiffies() which expects unsigned int only.
Signed-off-by: NMichal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170407133212.174608-1-michal.wajdeczko@intel.comReviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Since we reuse the same values for each fw_domain, move them onto
uncore.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170323101944.21627-6-chris@chris-wilson.co.uk

We can relax the requirement upon ourselves that the forcewake is
released immediately and just allow it to occur naturally following our
mmio request.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170323101944.21627-5-chris@chris-wilson.co.uk

Use find-first-set bitop to quickly scan through the fw_domains mask and
skip iterating over unused domains.

v2: Move the WARN into the caller, to prevent compiler warnings in
normal builds.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170323101944.21627-4-chris@chris-wilson.co.uk

In the next patch we will begin to sanity check that we do not attempt
to obtain the forcewake on an unsupport domain. However, that is exactly
what we do during reset of the fw_domains - rectify it before it explodes.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170323101944.21627-3-chris@chris-wilson.co.uk

In the next patch we will begin to sanity check that we do not attempt
to obtain the forcewake on an unsupport domain. However, that is exactly
what we do during our actual initialisation of fw_domains - rectify it
before it explodes.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170323101944.21627-2-chris@chris-wilson.co.uk

Pass along the drm_i915_private pointer from the caller, rather than
looking it up from each fw_domain during fw_domains_get/_put. This
allows us to then eliminate the backpointer, in exchange for a more
complicated unwrapping procedure in the rare
intel_uncore_fw_release_timer().
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170323101944.21627-1-chris@chris-wilson.co.uk

If we avoid initializing forcewake domains when running as
a guest, and also use gen2 mmio accessors in that case, we
can avoid the timer traffic and any looping through the
forcewake code which is currently just so it can end up in
the no-op forcewake implementation.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Weinan Li <weinan.z.li@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Tested-by: NTerrence Xu <terrence.xu@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170310095747.12258-1-tvrtko.ursulin@linux.intel.com
[tursulin: commit spelling fix]

In commit 003342a5 ("drm/i915: Keep track of active
forcewake domains in a bitmask") I forgot to adjust the
newly introduce fw_domains_active state across reset.

This caused the assert_forcewakes_inactive to trigger
during suspend and resume if there were user held
forcewakes.

v2: Bitmask checks are required since vfuncs are not
    always present.

v3: Move bitmask tracking to get/put vfunc for simplicity.
    (Chris Wilson)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Fixes: 003342a5 ("drm/i915: Keep track of active forcewake domains in a bitmask")
Testcase: igt/drv_suspend/forcewake
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: "Paneri, Praveen" <praveen.paneri@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Daniel Vetter <daniel.vetter@intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: intel-gfx@lists.freedesktop.org
Cc: v4.10+ <stable@vger.kernel.org>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/20170310093249.4484-1-tvrtko.ursulin@linux.intel.com
(cherry picked from commit b8473050)
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

In commit 003342a5 ("drm/i915: Keep track of active
forcewake domains in a bitmask") I forgot to adjust the
newly introduce fw_domains_active state across reset.

This caused the assert_forcewakes_inactive to trigger
during suspend and resume if there were user held
forcewakes.

v2: Bitmask checks are required since vfuncs are not
    always present.

v3: Move bitmask tracking to get/put vfunc for simplicity.
    (Chris Wilson)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Fixes: 003342a5 ("drm/i915: Keep track of active forcewake domains in a bitmask")
Testcase: igt/drv_suspend/forcewake
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: "Paneri, Praveen" <praveen.paneri@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Daniel Vetter <daniel.vetter@intel.com>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: intel-gfx@lists.freedesktop.org
Cc: v4.10+ <stable@vger.kernel.org>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/20170310093249.4484-1-tvrtko.ursulin@linux.intel.com

assert_spin_locked() becomes an unconditionally compiled BUG_ON(),
adding debug code right into the heart of critical routines like
interrupt handlers.

   text	   data	    bss	    dec	    hex
1296480	  19944	   2272	1318696	 141f28	before (lockdep disabled)
1295984	  19944	   2272	1318200	 141d38	after

1336261	  21139	   3208	1360608	 14c2e0	before (lockdep enabled)
1339920	  21139	   3208	1364267	 14d12b	after

Small saving for release; hopefully more instructive in debug.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170302132801.599-1-chris@chris-wilson.co.ukReviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>

Listen for PMIC bus access notifications and get FORCEWAKE_ALL while
the bus is accessed to avoid needing to do any forcewakes, which need
PMIC bus access, while the PMIC bus is busy:

This fixes errors like these showing up in dmesg, usually followed
by a gfx or system freeze:

[drm:fw_domains_get [i915]] *ERROR* render: timed out waiting for forcewake ack request.
[drm:fw_domains_get [i915]] *MEDIA* render: timed out waiting for forcewake ack request.
i2c_designware 808622C1:06: punit semaphore timed out, resetting
i2c_designware 808622C1:06: PUNIT SEM: 2
i2c_designware 808622C1:06: couldn't acquire bus ownership

Downside of this approach is that it causes wakeups whenever the PMIC
bus is accessed. Unfortunately we cannot simply wait for the PMIC bus
to go idle when we hit a race, as forcewakes may be done from interrupt
handlers where we cannot sleep to wait for the i2c PMIC bus access to
finish.

Note that the notifications and thus the wakeups will only happen on
baytrail / cherrytrail devices using PMICs with a shared i2c bus for
P-Unit and host PMIC access (i2c busses with a _SEM method in their
APCI node), e.g. an axp288 PMIC.

I plan to write some patches for drivers accessing the PMIC bus to
limit their bus accesses to a bare minimum (e.g. cache registers, do not
update battery level more often then 4 times a minute), to limit the
amount of wakeups.

BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=155241Signed-off-by: NHans de Goede <hdegoede@redhat.com>
Tested-by: Ntagorereddy <tagore.chandan@gmail.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
[danvet: Wiggle in conflicts.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Rename intel_uncore_early_sanitize to intel_uncore_resume, dropping the
(always true) restore_forcewake argument and add a new intel_uncore_resume
function to replace the intel_uncore_forcewake_reset(dev_priv, false)
calls done from the suspend / runtime_suspend functions and make
intel_uncore_forcewake_reset private.

This is a preparation patch for adding PMIC bus access notifier support.

BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=155241Signed-off-by: NHans de Goede <hdegoede@redhat.com>
Tested-by: Ntagorereddy <tagore.chandan@gmail.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20170210102802.20898-12-hdegoede@redhat.com

Now that the kselftest infrastructure exists, put it to use and add to
it the existing consistency checks on the fw register lookup tables.

v2: s/tabke/table/
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NMatthew Auld <matthew.auld@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170213171558.20942-22-chris@chris-wilson.co.uk

The only difference for the more recent of those macros is the version
of the *_reg_<read/write>_fw_domains function. Passing the function
prefix in allows us to re-use the same macro to generate functions for
different GENs and will make it easier to add new accessors in the future

Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NDaniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1486171409-21542-1-git-send-email-daniele.ceraolospurio@intel.comReviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

Having converted the force_wake_get/_put routines for a vGPU to be no-op,
we can use the common mmio accessors and remove our specialised routines
that simply skipped the calls to control force_wake.
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1485408228-12932-1-git-send-email-weinan.z.li@intel.comSigned-off-by: NChris Wilson <chris@chris-wilson.co.uk>

For a virtualized GPU, the host maintains the forcewake state on the real
device. As we don't control forcewake ourselves, we can simply set
force_wake_get() and force_wake_put() to be no-ops. By setting the vfuncs,
we adjust both the manual control of forcewake and around the mmio
accessors (making our vgpu specific mmio routines redundant and to be
removed in the next patch).
Signed-off-by: NWeinan Li <weinan.z.li@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1485408013-12780-1-git-send-email-weinan.z.li@intel.comSigned-off-by: NChris Wilson <chris@chris-wilson.co.uk>

ktime_set(S,N) was required for the timespec storage type and is still
useful for situations where a Seconds and Nanoseconds part of a time value
needs to be converted. For anything where the Seconds argument is 0, this
is pointless and can be replaced with a simple assignment.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>

Add WARN_ON to find_fw_domain to registers related to uninitialized
hardware.

v2:
- Print the uninitialized domains and register (Chris)

Cc: Imre Deak <imre.deak@intel.com>
Cc: Wang Elaine <elaine.wang@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1481120559-17413-1-git-send-email-joonas.lahtinen@linux.intel.com

Pineview deserves to use its own platform enum (which was already added,
unused, previously). IS_G33() no longer matches Pineview, and gets
replaced by IS_G33() || IS_PINEVIEW() or equivalent. Pineview is no
longer an outlier among platform definitions.

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1481143689-19672-1-git-send-email-jani.nikula@intel.com

The check in __intel_uncore_early_sanitize() to disable decoupled mmio
would disable it for every platform that is not broxton. While that's
not a problem now since only broxton supports that, simply setting
.has_decoupled_mmio in a new platform's device info wouldn't suffice. So
avoid future confusion and change the workaround to only change the
value of has_decoupled_mmio for broxton.

v2: git add compile fix. (Ander)

Cc: Praveen Paneri <praveen.paneri@intel.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Signed-off-by: NAnder Conselvan de Oliveira <ander.conselvan.de.oliveira@intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1479993807-29353-1-git-send-email-ander.conselvan.de.oliveira@intel.com

We have to make sure there are no holes in the table in Gen9.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Suggested-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1479388435-12062-1-git-send-email-tvrtko.ursulin@linux.intel.com

A bunch of source files with just a few instances of the
incorrect INTEL_INFO use.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Commit 0dd356bb ("drm/i915: Eliminate Gen9 special case")
accidentaly dropped a MMIO range between 0xc000 to 0xcfff out
of the blitter forcewake domain. Fix it.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Fixes: 0dd356bb ("drm/i915: Eliminate Gen9 special case")
Reported-by: NDaniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1479373363-16528-1-git-send-email-tvrtko.ursulin@linux.intel.com

Decoupled MMIO is an alternative way to access forcewake domain
registers, which requires less cycles for a single read/write and
avoids frequent software forcewake.
This certainly gives advantage over the forcewake as this new
mechanism “decouples” CPU cycles and allow them to complete even
when GT is in a CPD (frequency change) or C6 state.

This can co-exist with forcewake and we will continue to use forcewake
as appropriate. E.g. 64-bit register writes to avoid writing 2 dwords
separately and land into funny situations.

v2:
- Moved platform check out of the function and got rid of duplicate
 functions to find out decoupled power domain (Chris)
- Added a check for forcewake already held and skipped decoupled
 access (Chris)
- Skipped writing 64 bit registers through decoupled MMIO (Chris)

v3:
- Improved commit message with more info on decoupled mmio (Tvrtko)
- Changed decoupled operation to enum and used u32 instead of
 uint_32 data type for register offset (Tvrtko)
- Moved HAS_DECOUPLED_MMIO to device info (Tvrtko)
- Added lookup table for converting fw_engine to pd_engine (Tvrtko)
- Improved __gen9_decoupled_read and __gen9_decoupled_write
 routines (Tvrtko)

v4:
- Fixed alignment and variable names (Chris)
- Write GEN9_DECOUPLED_REG0_DW1 register in just one go (Zhe Wang)

v5:
- Changed HAS_DECOUPLED_MMIO() argument name to dev_priv (Tvrtko)
- Sanitize info->had_decoupled_mmio at init (Chris)
Signed-off-by: NZhe Wang <zhe1.wang@intel.com>
Signed-off-by: NPraveen Paneri <praveen.paneri@intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1479230360-22395-1-git-send-email-praveen.paneri@intel.com

When we enable the per-register access mmiodebug, it is to detect which
access is illegal. Reporting on earlier untraced access outside of the
mmiodebug does not help debugging (as the suspicion is immediately put
upon the current register which is not at fault)!

References: https://bugs.freedesktop.org/show_bug.cgi?id=97985Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Cc: stable@vger.kernel.org
Link: http://patchwork.freedesktop.org/patch/msgid/20161003124516.12388-1-chris@chris-wilson.co.uk
(cherry picked from commit dda96033)
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

Instead of using bsearch library function make a local generator
macro out of it so the comparison callback can be inlined.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Suggested-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1475569769-31108-1-git-send-email-tvrtko.ursulin@linux.intel.com

Simply replace the linear search with the kernel's binary
search implementation. There is only six registers currently
in that table so this may not be that interesting. It adds a
function call so hopefully remains performance neutral for now.

v2: No need for manual conversion to bool for return.
    (Joonas Lahtinen)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Also verify the order at runtime. This was we can start using
binary search on it in a following patch.

v2: Add comment on the sorted array and only check it when
    debug option is enabled.

v3: Use IS_ENABLED. (Chris Wilson)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> (v1)
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

We notice two identical copies of the shadow register table and
following from that removal can also unify CHV and Gen9 write
mmio functions and macros into a single implementation.

v2: Name fwtable consistently and use HAS_FWTABLE. (Joonas Lahtinen)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>