Supposedly we would want to get the PWM output as close as possible to
the target, so let's round to closest.

Cc: Jani Nikula <jani.nikula@linux.intel.com>
Suggested-by: NJani Nikula <jani.nikula@linux.intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456932138-14004-7-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NJani Nikula <jani.nikula@intel.com>

Now that the mess with AUX clock divder rounding is sorted out and
we have both cdclk and rawclk cached in dev_priv, we can clean up
the .get_aux_clock_divider() functions a bit.

The main thing here is just calling ilk_get_aux_clock_divider()
from hsw_get_aux_clock_divider() except for the LPT:H special
case.

We could go further and call g4x_get_aux_clock_divider() from
ilk_get_aux_clock_divider() for the PCH ports, but I'm sure Jani
would object, so leave that be.

While at it repeat the comment where the AUX clock comes from
in ilk_get_aux_clock_divider().
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456932138-14004-6-git-send-email-ville.syrjala@linux.intel.com

Currently we assume that hrawclk is 200MHz on VLV/CHV. That should
be true always, but just to avoid such asumptions we can read out the
actual frequency from CCK.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456932138-14004-5-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NImre Deak <imre.deak@intel.com>

With the hrawclk frequency cached in dev_priv, we can simply use
g4x_get_aux_clock_divider() for VLV/CHV.

v2: Rebase due to IS_VALLYVIEW vs. IS_CHERRYVIEW split
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456932138-14004-4-git-send-email-ville.syrjala@linux.intel.com

g4x is the first platform with DP support, so let's name the relevant
functions as g4x_ instead i9xx_ to avoid confusion.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456932138-14004-3-git-send-email-ville.syrjala@linux.intel.com

Generalize rawclk handling by storing it in dev_priv.

Presumably our hrawclk readout works at least for CTG and ELK
since we've been using it for DP AUX on those platforms. There
are no real docs anymore after configdb vanished, so the only
reference is the public CTG GMCH spec. What bits are listed in
that doc match our code. The ELK GMCH spec have no relevant
details unfortunately.

The PNV situation is less clear. Starting from
commit aa17cdb4 ("drm/i915: initialize backlight max from VBT")
we assume that the CTG/ELK hrawclk readout works for PNV as well.
At least the results *seem* reasonable for one PNV machine (Lenovo
Ideapad S10-3t). Sadly the PNV GMCH spec doesn't have the goods on
the relevant register either.

So let's keep assuming it works for PNV,ELK,CTG and read it out on
those platforms. G33 also has hrawclk according to some notes
in BSpec, but we don't actually need it for anything, so let's not
even try to read it out there.

v2: Rebase due to IS_VALLYVIEW vs. IS_CHERRYVIEW split
    Use KHz() all over, and kill off a few useless temp variables
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456932138-14004-2-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NJani Nikula <jani.nikula@intel.com>

Looks like this code does not need to wait atomically since it
otherwise takes the mutex.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1457015805-23742-1-git-send-email-tvrtko.ursulin@linux.intel.com

Currently the wait_for_atomic_us only allows for a jiffie
timeout granularity which is not nice towards callers
requesting small micro-second timeouts.

Re-implement it so micro-second timeout granularity is really
supported and not just in the name of the macro.

This has another beneficial side effect that it improves
"gem_latency -n 100" results by approximately 2.5% (throughput
and latencies) and 3% (CPU usage). (Note this improvement is
relative to not yet merged execlist lock uncontention patch
which moves the CSB MMIO outside this lock.)

It also shrinks some hot functions like fw_domains_get by a
tiny 3%.

v2:
  * Warn when used from non-atomic context (if possible).
  * Warn on too long atomic waits.

v3:
  * Added comment explaining CONFIG_PREEMPT_COUNT.
  * Fixed pre-processor indentation.
  (Chris Wilson)

v4:
 * Commit msg update (gem_latency) and rebase.

v5:
 * Commit message re-wording.
 * Added comment about no need for double cond check. (Chris Wilson)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

v2: Added a submenu based on an idea by Chris Wilson.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

I do not see that this needs to be done atomically and up to
one second is quite a long time to busy loop.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

This is for callers who want micro-second precision but are not
waiting from the atomic context.

v2:
  * Fix atomic waits. (Dave Gordon)
  * Use USEC_PER_SEC and USEC_PER_MSEC. (Chris Wilson)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Dave Gordon <david.s.gordon@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

The MIPI clock calculations for the addtional clock
are revised from B0 stepping onwards, the bit definitions
have changed compared to old stepping.

v2: Fixing compilation warning.
v3: Retained the old Macros (Jani)
Signed-off-by: NDeepak M <m.deepak@intel.com>
Tested-by: Ramalingam C <ramalingam.c@intel.com> # BXT-T with Tianma panel
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455556437-29267-1-git-send-email-m.deepak@intel.com

Only planes that are part of the state should be used for recalculating
watermarks. For planes not part of the state the previous patch allows
us to re-use the old values since they're calculated even for levels
that are not actively used.

Changes since v1:
- Remove big if from intel_crtc_atomic_check.
- Remove extra newline.
- Remove memset in ilk_compute_pipe_wm.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456826842-32553-2-git-send-email-maarten.lankhorst@linux.intel.comReviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>

As Paulo has noted we can help bisectability by separating computing
watermarks on a noop in 2 separate commits.

This patch no longer clears the crtc watermark state, but recalculates
it completely. Regardless whether a level is used the full values for
each level are calculated. If a level is invalid wm[level].enable is
unset.

Changes since v1:
- Only call ilk_validate_wm_level when level <= usable_level. (Ville)
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/56D6D09E.5040007@linux.intel.comReviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>

This function returns an int, but when ilk_validate_pipe_wm fails it
returns false, which is 0 (success). As a result invalid watermarks
are applied, while they should have been rejected.

Fix this by returning -EINVAL.

Fixes: ed4a6a7c ("drm/i915: Add two-stage ILK-style watermark programming (v11)")
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456918563-28696-1-git-send-email-maarten.lankhorst@linux.intel.comReviewed-by: NMatt Roper <matthew.d.roper@intel.com>

This changes ensures device is active when frequency limits are changed.
This is needed as we are writing to register RPNSWREQ in intel_set_rps.
If not done, might lead to undesired errors like:
[ 1965.189137] [drm:fw_domains_get] *ERROR* blitter: timed out waiting for forcewake ack to clear.

v2: Added elaborate commit message. (Jani)
    Fixing RPM reference drop in early exit paths. (Ville)
Signed-off-by: NSagar Arun Kamble <sagar.a.kamble@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1454951831-11778-1-git-send-email-sagar.a.kamble@intel.comSigned-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>

   commit e5756c10
   Author: Imre Deak <imre.deak@intel.com>
   Date:   Fri Aug 14 18:43:30 2015 +0300

       drm/i915/bxt: don't allow cached GEM mappings on A stepping

Added an exception of disallowing snooping for Broxton A
stepping hardware but userptr was still enabling it regardless.

Move the check to HAS_SNOOP now that it is used from multiple
call sites and use it.

v2: Userptr cannot be supported when it cannot be coherent and
    generalize the code better. (Chris Wilson)

v3: Make has_snoop true only when !has_llc. (Chris Wilson)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1456920631-34302-1-git-send-email-tvrtko.ursulin@linux.intel.com

This fixes the IGT test, which interprets unknown status as failed to
acquire load detect pipe.

Cc: Gabriel Feceoru <gabriel.feceoru@intel.com>
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456848241-6431-1-git-send-email-maarten.lankhorst@linux.intel.comReviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>

Disabling the DC states when it's already disabled is a valid scenario,
for example during HW state sanitization during driver loading and
resuming or when DC states are disabled via the i915.enable_dc or
disable_power_well option.

CC: Patrik Jakobsson <patrik.jakobsson@linux.intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NPatrik Jakobsson <patrik.jakobsson@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456778945-5411-4-git-send-email-imre.deak@intel.com

If power well support is disabled via the i915.disable_power_well module
option we should never enable DC states. Currently we would enable DC
states even in this case during system suspend, where we need to disable
all power wells regardless of the disable_power_well option.

CC: Patrik Jakobsson <patrik.jakobsson@linux.intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NPatrik Jakobsson <patrik.jakobsson@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456778945-5411-3-git-send-email-imre.deak@intel.com

We can simplify the conditions selecting the target DC state during
runtime by calculating the allowed DC states in advance during driver
loading. This also makes it easier to disable DC states depending on the
i915.disable_power_well module option, added in the next patch.

v2:
- Print a debug message if the requested max DC value was adjusted due
  to a platform limit. Also debug print the calculated mask value. (Patrik)

CC: Patrik Jakobsson <patrik.jakobsson@linux.intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NPatrik Jakobsson <patrik.jakobsson@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456778945-5411-2-git-send-email-imre.deak@intel.com

During system suspend we need to first disable power wells then
unitialize the display core. In case power well support is disabled we
did this in the wrong order, so fix this up.

Fixes: d314cd43 ("drm/i915: fix handling of the disable_power_well module option")
CC: stable@vger.kernel.org
CC: Patrik Jakobsson <patrik.jakobsson@linux.intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NPatrik Jakobsson <patrik.jakobsson@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456778945-5411-1-git-send-email-imre.deak@intel.com

execute during context save/restore, good to have them in error state.

v2: use wa_ctx->size and print only size values (Mika)
v3: simplify conditions when recording and freeing object (Chris)
v4: resolve checkpatch errors (Tvrtko)

Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NArun Siluvery <arun.siluvery@linux.intel.com>
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456831476-10782-1-git-send-email-arun.siluvery@linux.intel.com

LPT/WPT-H are limited to max 180 MHz CRT dotclock. Most other platforms
have a limit of 350 MHz. Supposedly gen3 and gen4 go up to 400 MHz.

VLV is a bit special since the docs are poor. Supposedly the DAC
would be good up to 355 MHz, but currently we limit the DPLL to
270 MHz, so we'll have to limit the port clock to the same unless
we change the DPLL limits.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455738073-14502-7-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NImre Deak <imre.deak@intel.com>

Rather than assume the VGA dotclock is really the FDI based thing,
let's read out the real thing via iclkip, and after readout it'll
get to compare it with the FDI based number to make sure they're
in sync.

Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455738073-14502-6-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NImre Deak <imre.deak@intel.com>

The reason for spcial casing 20MHz in the iclkip calculations is that
it would overflow the 7 bit divisor value. Let's rewrite the special
case to check for just that, and bump up auxdiv when needed. This makes
the code work for freqeuencies close to but not exactly 20MHz. The real
lower limit for auxdiv=0 is actually:
172800000/(0x7f+2)*64)=~20930 kHz, and below that we must resort to
auxdiv=1.

Actually this is all very theoretical since we limit the dotclock to
min 25MHz with CRT on all platforms. 25Mhz is actually the documented
limit in Bspec, so it seems we ought to never need to worry about the
auxdiv=1 case. But no harm in having it.

Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455738073-14502-5-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>

Instead of assuming we've correctly set up SPLL to run at 270Mhz for
FDI, let's use the port_clock from pipe_config which should be what
we want. This would catch problems if someone misconfigures SPLL for
whatever reason.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455738073-14502-4-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NImre Deak <imre.deak@intel.com>

Currently we check if the encoder's idea of dotclock agrees with what
we calculated based on the FDI parameters. We do this in the encoder
.get_config() hooks, which isn't so nice in case the BIOS (or some other
outside party) made a mess of the state and we're just trying to take
over.

So as a prep step to being able sanitize such a bogus state, move the
the sanity check to just after we've read out the entire state. If
we then need to sanitize a bad state, it should be easier to move the
sanity check to occur after sanitation instead of before it.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455738073-14502-3-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NImre Deak <imre.deak@intel.com>

On HSW/BDW ddi_pll_sel is the actual register value. Let's dump
it in hex so that people migth actually understand what it says.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455738073-14502-2-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NImre Deak <imre.deak@intel.com>

Instead of repopulatin the rotation_info struct for the fb every time
we try to use the fb, we can just populate it once when creating the fb,
and later we can just copy the pre-populate struct into the gtt_view.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1455569699-27905-10-git-send-email-ville.syrjala@linux.intel.com

rotate_pages() checks to see if it got called with a NULL sg, and then
goes to extract it from sg->sgl. It always gets called with a NULL sg
for the first plane, so moving the initial 'sg=st->sgl' assignment out
into intel_rotate_fb_obj_pages() seems less special-casey.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1455569699-27905-9-git-send-email-ville.syrjala@linux.intel.com

Throw out a bunch of unnecessary stuff from struct intel_rotation_info,
and pull most of the remaining stuff to live under an array of
per-color plane sub-structures.

What still remains outside the sub-structure will be reorgranized later
as well, but that requires more work elsewhere so leave it be for now.

v2: Split the vma size == luma+chroma size fix to prep patch (Daniel)
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> (v1)
Link: http://patchwork.freedesktop.org/patch/msgid/1455569699-27905-8-git-send-email-ville.syrjala@linux.intel.com

intel_compute_page_offsets() gets passed a bunch of the framebuffer
metadate sepearately. Just pass the framebuffer itself to make life
simpler for the caller, and make it less likely they would make a
mistake in the order of the arguments (as most as just unsigned ints and
such).

We still pass the pitch explicitly since for 90/270 degree rotation
the caller has to pass in the right thing.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1455569699-27905-7-git-send-email-ville.syrjala@linux.intel.com

intel_pin_and_fence_fb_obj() only needs the framebuffer, and the desird
rotation (to find the right GTT view for it), so no need to pass all
kinds of plane stuff.

The main motivation is to get rid of the uggy NULL plane_state handling
due to fbdev.

v2: Add a note why I really want this
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Grumpily-Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1455569699-27905-6-git-send-email-ville.syrjala@linux.intel.com

SKL+ needs >4K alignment for tiled surfaces, so make
intel_compute_page_offset() handle it.

The way we do it is first we compute the closest tile boundary
as before, and then figure out how many tiles we need to go
to reach the desired alignment. The difference in the offset
is then added into the x/y offsets.

v2: Be less confusing wrt. units (pixels vs. bytes) (Daniel)
v3: Use u32 for offsets
    Have intel_adjust_tile_offset() return the new offset (will be
    useful later)
    Add an offset_aligned variable (Daniel)
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455569699-27905-5-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The page aligned surface address calculation needs to know which way
things are rotated. The contract now says that the caller must pass the
rotate x/y coordinates, as well as the tile_height aligned stride in
the tile_height direction. This will make it fairly simple to deal with
90/270 degree rotation on SKL+ where we have to deal with the rotated
view into the GTT.

v2: Pass rotation instead of bool even thoughwe only care about 0/180 vs. 90/270
v3: Introduce intel_tile_dims(), and don't mix up different units so much
v4: Unconfuse bytes vs. pixels even more
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455569699-27905-4-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Make if clear whether we're talking tile widths in bytes or in pixels.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455569699-27905-3-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The size of the rotated ggtt mapping ought to include the size of the
chroma plane as well. Not a huge deal since we don't expose NV12 (or any
pother planar format for that matter) yet.

Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Fixes: 89e3e142 ("drm/i915: Support NV12 in rotated GGTT mapping")
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455569699-27905-2-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>

Not all platforms set this callback, so NULL check it before calling it.

v2:
- Call intel_update_watermarks() on HSW+ where the callback is not set.
  (Matt)

CC: Matt Roper <matthew.d.roper@intel.com>
Fixes: commit ed4a6a7c ("drm/i915: Add two-stage ILK-style watermark programming (v11)")
Signed-off-by: NImre Deak <imre.deak@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456776633-3401-1-git-send-email-imre.deak@intel.comReviewed-by: NMatt Roper <matthew.d.roper@intel.com>

Assorted changes in the areas of code cleanup, reduction of
invariant conditional in the interrupt handler and lock
contention and MMIO access optimisation.

 * Remove needless initialization.
 * Improve cache locality by reorganizing code and/or using
   branch hints to keep unexpected or error conditions out
   of line.
 * Favor busy submit path vs. empty queue.
 * Less branching in hot-paths.

v2:

 * Avoid mmio reads when possible. (Chris Wilson)
 * Use natural integer size for csb indices.
 * Remove useless return value from execlists_update_context.
 * Extract 32-bit ppgtt PDPs update so it is out of line and
   shared with two callers.
 * Grab forcewake across all mmio operations to ease the
   load on uncore lock and use chepear mmio ops.

v3:

 * Removed some more pointless u8 data types.
 * Removed unused return from execlists_context_queue.
 * Commit message updates.

v4:
 * Unclumsify the unqueue if statement. (Chris Wilson)
 * Hide forcewake from the queuing function. (Chris Wilson)

Version 3 now makes the irq handling code path ~20% smaller on
48-bit PPGTT hardware, and a little bit less elsewhere. Hot
paths are mostly in-line now and hammering on the uncore
spinlock is greatly reduced together with mmio traffic to an
extent.

Benchmarking with "gem_latency -n 100" (keep submitting
batches with 100 nop instruction) shows approximately 4% higher
throughput, 2% less CPU time and 22% smaller latencies. This was
on a big-core while small-cores could benefit even more.

Most likely reason for the improvements are the MMIO
optimization and uncore lock traffic reduction.

One odd result is with "gem_latency -n 0" (dispatching empty
batches) which shows 5% more throughput, 8% less CPU time,
25% better producer and consumer latencies, but 15% higher
dispatch latency which is yet unexplained.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1456505912-22286-1-git-send-email-tvrtko.ursulin@linux.intel.com