Calculate the DDB blocks needed to satisfy the current atomic
transaction at atomic check time.  This is a prerequisite to calculating
SKL watermarks during the 'check' phase and rejecting any configurations
that we can't find valid watermarks for.

Due to the nature of DDB allocation, it's possible for the addition of a
new CRTC to make the watermark configuration already in use on another,
unchanged CRTC become invalid.  A change in which CRTC's are active
triggers a recompute of the entire DDB, which unfortunately means we
need to disallow any other atomic commits from racing with such an
update.  If the active CRTC's change, we need to grab the lock on all
CRTC's and run all CRTC's through their 'check' handler to recompute and
re-check their per-CRTC DDB allocations.

Note that with this patch we only compute the DDB allocation but we
don't actually use the computed values during watermark programming yet.
For ease of review/testing/bisecting, we still recompute the DDB at
watermark programming time and just WARN() if it doesn't match the
precomputed values.  A future patch will switch over to using the
precomputed values once we're sure they're being properly computed.

Another clarifying note:  DDB allocation itself shouldn't ever fail with
the algorithm we use today (i.e., we have enough DDB blocks on BXT to
support the minimum needs of the worst-case scenario of every pipe/plane
enabled at full size).  However the watermarks calculations based on the
DDB may fail and we'll be moving those to the atomic check as well in
future patches.

v2:
 - Skip DDB calculations in the rare case where our transaction doesn't
   actually touch any CRTC's at all.  Assuming at least one CRTC state
   is present in our transaction, then it means we can't race with any
   transactions that would update dev_priv->active_crtcs (which requires
   _all_ CRTC locks).

v3:
 - Also calculate DDB during initial hw readout, to prevent using
   incorrect bios values. (Maarten)

v4:
 - Use new distrust_bios_wm flag instead of skip_initial_wm (which was
   never actually set).
 - Set intel_state->active_pipe_changes instead of just realloc_pipes

Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Lyude Paul <cpaul@redhat.com>
Cc: Radhakrishna Sripada <radhakrishna.sripada@intel.com>
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1463061971-19638-10-git-send-email-matthew.d.roper@intel.com

SKL-style platforms can't fully trust the watermark/DDB settings
programmed by the BIOS and need to do extra sanitization on their first
atomic update.  Add a flag to dev_priv that is set during hardware
readout and cleared at the end of the first commit.

Note that for the somewhat common case where everything is turned off
when the driver starts up, we don't need to bother with a recompute...we
know exactly what the DDB should be (all zero's) so just setup the DDB
directly in that case.

v2:
 - Move clearing of distrust_bios_wm up below the swap_state call since
   it's a more natural / self-explanatory location.  (Maarten)
 - Use dev_priv->active_crtcs to test whether any CRTC's are turned on
   during HW WM readout rather than trying to count the active CRTC's
   again ourselves.  (Maarten)

Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1463061971-19638-9-git-send-email-matthew.d.roper@intel.com

We eventually want to calculate watermark values at atomic 'check' time
instead of atomic 'commit' time so that any requested configurations
that result in impossible watermark requirements are properly rejected.
The first step along this path is to allocate the DDB at atomic 'check'
time.  As we perform this transition, allow the main allocation function
to operate successfully on either an in-flight state or an
already-commited state.  Once we complete the transition in a future
patch, we'll come back and remove the unnecessary logic for the
already-committed case.

v2: Rebase/refactor; we should no longer need to grab extra plane states
    while allocating the DDB since we can pull cached data rates and
    minimum block counts from the CRTC state for any planes that aren't
    being modified by this transaction.

v3:
 - Simplify memsets to clear DDB plane entries.  (Maarten)
 - Drop a redundant memset of plane[pipe][PLANE_CURSOR] that was added
   by an earlier Coccinelle patch.  (Maarten)
 - Assign *num_active at the top of skl_ddb_get_pipe_allocation_limits()
   so that no code paths return without setting it.  (kbuild robot)

Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1463061971-19638-8-git-send-email-matthew.d.roper@intel.com

This will eventually allow us to re-use old values without
re-calculating them for unchanged planes (which also helps us avoid
re-grabbing extra plane states).

v2:
 -  Drop unnecessary memset's; they were meant for a later patch (which
    got reworked anyway to not need them, but were mis-rebased into this
    one.  (Maarten)

Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1463061971-19638-6-git-send-email-matthew.d.roper@intel.com

Our skl_get_total_relative_data_rate() function gets passed a crtc state
object to calculate the data rate for, but it currently always looks
up the committed plane states that correspond to that CRTC.  Let's
check whether the CRTC state is an in-flight state (meaning
cstate->state is non-NULL) and if so, use the corresponding in-flight
plane states.

We'll soon be using this function exclusively for in-flight states; at
that time we'll be able to simplify the function a bit, but for now we
allow it to be used in either mode.

v2:
 - Rebase on top of changes to cache plane data rates.
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1463061971-19638-5-git-send-email-matthew.d.roper@intel.com

This will be important when we start calculating CRTC data rates for
in-flight CRTC states since it will allow us to calculate the total data
rate without needing to grab the plane state for any planes that aren't
updated by the transaction.
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1463061971-19638-4-git-send-email-matthew.d.roper@intel.com

When we added atomic watermarks, we added a new display vfunc
'compute_pipe_wm' that is used to compute any pipe-specific watermark
information that we can at atomic check time.  This was a somewhat poor
naming choice since we already had a 'skl_compute_pipe_wm' function that
doesn't quite fit this model --- the existing SKL function is something
that gets used at atomic commit time, after the DDB allocation has been
determined.  Let's rename the existing SKL function to avoid confusion.
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1463061971-19638-3-git-send-email-matthew.d.roper@intel.com

Reorganize the nested structures and unions we have for pipe watermark
data in intel_crtc_state so that platform-specific data can be added in
a more sensible manner (and save a bit of memory at the same time).

The change basically changes the organization from:

        union {
                struct intel_pipe_wm ilk;
                struct intel_pipe_wm skl;
        } optimal;

        struct intel_pipe_wm intermediate /* ILK-only */

to

        union {
                struct {
                        struct intel_pipe_wm intermediate;
                        struct intel_pipe_wm optimal;
                } ilk;

                struct {
                        struct intel_pipe_wm optimal;
                } skl;
        }

There should be no functional change here, but it will allow us to add
more platform-specific fields going forward (and more easily extend to
other platform types like VLV).

While we're at it, let's move the entire watermark substructure out to
its own structure definition to make the code slightly more readable.
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1463061971-19638-2-git-send-email-matthew.d.roper@intel.com

This way optimization from a previous patch works even better.

v2: Rebase.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>

Pass drm_i915_private to the uncore init/fini routines and their
subservients as it is their native type.

   text    data     bss     dec     hex filename
6309978 3578778  696320 10585076         a183f4 vmlinux
6309530 3578778  696320 10584628         a18234 vmlinux

a modest 400 bytes of saving, but 60 lines of code deleted!
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1462885804-26750-1-git-send-email-chris@chris-wilson.co.uk

Use the cdclk we're going to be using when the pipe gets enabled to
compute the IPS linetime watermark. The current cdclk frequency is
irrelevant at this point since it can still change.

Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1461940278-17122-1-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>

   text	   data	    bss	    dec	    hex	filename
6309351	3578714	 696320	10584385	 a18141	vmlinux
6308391	3578714	 696320	10583425	 a17d81	vmlinux

Almost 1KiB of code reduction.

v2: More s/INTEL_INFO()->gen/INTEL_GEN()/ and IS_GENx() conversions

   text	   data	    bss	    dec	    hex	filename
6304579	3578778	 696320	10579677	 a16edd	vmlinux
6303427	3578778	 696320	10578525	 a16a5d	vmlinux

Now over 1KiB!
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1462545621-30125-3-git-send-email-chris@chris-wilson.co.uk

I have noticed some of our interrupt handlers use both dev and
dev_priv while they could get away with only dev_priv in the
huge majority of cases.

Tidying that up had a cascading effect on changing functions
prototypes, so relatively big churn factor, but I think it is
for the better.

For example even where changes cascade out of i915_irq.c, for
functions prefixed with intel_, genX_ or <plat>_, it makes more
sense to take dev_priv directly anyway.

This allows us to eliminate local variables and intermixed usage
of dev and dev_priv where only one is good enough.

End result is shrinkage of both source and the resulting binary.

i915.ko:

 - .text         000b0899
 + .text         000b0619

Or if we look at the Gen8 display irq chain:

 -00000000000006ad t gen8_irq_handler
 +0000000000000663 t gen8_irq_handler
   -0000000000000028 T intel_opregion_asle_intr
   +0000000000000024 T intel_opregion_asle_intr
   -000000000000008c t ilk_hpd_irq_handler
   +000000000000007f t ilk_hpd_irq_handler
   -0000000000000116 T intel_check_page_flip
   +0000000000000112 T intel_check_page_flip
   -000000000000011a T intel_prepare_page_flip
   +0000000000000119 T intel_prepare_page_flip
   -0000000000000014 T intel_finish_page_flip_plane
   +0000000000000013 T intel_finish_page_flip_plane
   -0000000000000053 t hsw_pipe_crc_irq_handler
   +000000000000004c t hsw_pipe_crc_irq_handler
   -000000000000022e t cpt_irq_handler
   +0000000000000213 t cpt_irq_handler

So small shrinkage but it is all fast paths so doesn't harm.

Situation is similar in other interrupt handlers as well.

v2: Tidy intel_queue_rps_boost_for_request as well. (Chris Wilson)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

While browsing BSpec I bumped into a note saying we need to tune these
values based on actual measurements done after initial enabling. I've
checked that it indeed improves things on BXT. I haven't checked this on
CHV, but here it is if someone wants to give it a go.

v2:
- Add note about the discrepancy wrt. to the spec in the formula
  calculating the credit encodings. (Mika, Ville)
- Move the WA comment to the new function. (Ville)
v3:
- Keep the comment about the SQC WA in the caller. (Ville)

CC: Ville Syrjälä <ville.syrjala@linux.intel.com>
CC: Mika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1462280061-1457-4-git-send-email-imre.deak@intel.com

No need for hard-coding the register value, the corresponding fields are
defined properly in BSpec.

No functional change.

v2:
- Rebased on BXT L3 SQC tuning patch merged meanwhile.

CC: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com> (v1)
Link: http://patchwork.freedesktop.org/patch/msgid/1462280061-1457-3-git-send-email-imre.deak@intel.com

BSpec requires us to wait ~100 clocks before re-enabling clock gating,
so make sure we do this.

CC: stable@vger.kernel.org
CC: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1462280061-1457-2-git-send-email-imre.deak@intel.com

If we move the release of the GEM request (i.e. decoupling it from the
various lists used for client and context tracking) after it is complete
(either by the GPU retiring the request, or by the caller cancelling the
request), we can remove the requirement that the final unreference of
the GEM request need to be under the struct_mutex.

The careful reader may notice that one or two impossible NULL pointer
tests are dropped for readability. These pointers cannot be NULL since
they are assigned during request construction and never unset.

v2,v3: Rebalance execlists by moving the context unpinning.
v4: Rebase onto -nightly
v5: Avoid trying to rebalance execlist/GuC context pinning, leave that
to the next step
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1461833819-3991-21-git-send-email-chris@chris-wilson.co.uk

As a part of WaGsvDisableTurbo, Driver makes an early exit from the
Gen9 Turbo enabling function, so doesn't program the Turbo Control register.
But BIOS could leave the Hw Turbo as enabled, so need to explicitly clear
out the Control register just to avoid inconsitency with debugfs
interface, which will show Turbo as enabled only and that is not expected
after adding the WaGsvDisableTurbo. Apart from this there is no problem
even if the Turbo is left enabled in the Control register, as the Up/Down
interrupts would remain masked.

v2: Add explicit clearing of Turbo Control register to *_disable_rps()
    also for the similar consistency (Chris)
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NAkash Goel <akash.goel@intel.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1461350146-23454-2-git-send-email-akash.goel@intel.comSigned-off-by: NChris Wilson <chris@chris-wilson.co.uk>

On CHV GTFIFODBG has some read-only bits to indicate the number
of free FIFO entries. Ignore these when checking to see if any
of the sticky error bits are set.

This gets rid of these during device resume:
[drm:cherryview_enable_rps] GT fifo had a previous error 1080000

While at it, move the assignments out of the if().

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1460570970-14073-1-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The registers frobbed by vlv_init_display_clock_gating() libve inside
the disp2d power well, so frobbing them while the power well is down
results in unclaimed register access warning (and of course the values
won't stick). Let's do this setup after we know the power well is
enabled.

It's also worth noting that DSPCLK_GATE_D and CBR1_VLV lose their state
when the power well goes down, but fortunately the values we've been
writing are actually the reset defaults.

MI_ARB_VLV actually retains its value even if the power well was turned
off, we just can't access it while the power well is down.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=94164Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1460382992-28728-9-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NImre Deak <imre.deak@intel.com>

Once again ILK is unhappy if we clear out the LP1+ watermark levels
outright, and instead we must disable the levels we don't want while
still leaving the actual programmed watermark levels intact.

Fixes underruns on the already enabled pipe when programming watermarks
while enabling the second pipe.

Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Matt Roper <matthew.d.roper@intel.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=93787Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1459536799-18109-3-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NPatrik Jakobsson <patrik.jakobsson@linux.intel.com>

dev_priv is what the macro works hard to extract, pass it directly.

> sed 's/\([A-Z].*(dev_priv\)->dev)/\1)/g'

v2:
- Include all wrapper macros too (Chris)

v3:
- Include sed cmdline (Chris)

v4:
- Break long line
- Rebase

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1460016485-8089-1-git-send-email-joonas.lahtinen@linux.intel.com

Use plane size for relative data rate calculation. don't always use
pipe source width & height.
adjust height & width according to rotation.
use plane size for watermark calculations also.

v2: Address Matt's comments.
    Use intel_plane_state->visible to avoid divide-by-zero error.
    Where FB was present but not visible so causing total data rate to
    be zero, hence divide-by-zero.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=93917
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=94044
Cc: drm-intel-fixes@lists.freedesktop.org
Signed-off-by: NKumar, Mahesh <mahesh1.kumar@intel.com>
Reviewed-by: NMatt Roper <matthew.d.roper@intel.com>
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1459956399-1296-1-git-send-email-matthew.d.roper@intel.com

Currently we set the initial GPU frequency to min_freq_softlimit
on gen9, and to efficient_freq on VLV/CHV. On all the other platforms
we set it to idle_freq. Let's use idle_freq across the board to make
sure we don't waste power. This is especially relevant for VLV since
Vnn won't drop to minimum unless the GPU is at the minimum frequency.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1457120584-26080-3-git-send-email-ville.syrjala@linux.intel.comReviewed-by: NImre Deak <imre.deak@intel.com>

Extract the GPLL reference frequency from CCK and use it in the
GPU freq<->opcode conversions on VLV/CHV. This eliminates all the
assumptions we have about which divider is used for which czclk
frequency.

Note that unlike most clocks from CCK, the GPLL ref clock is a divided
down version of the CZ clock rather than the HPLL clock. CZ clock itself
is a divided down version of the HPLL clock though, so in effect it just
gets divided down twice.

While at it, throw in a few comments explaining the remaining constants
for anyone who later wants to compare this to the spreadsheets.

v2: Add slow/fast notes for CHV clocks (Imre)
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1457120584-26080-2-git-send-email-ville.syrjala@linux.intel.com
Reviewed-by: Imre Deak <imre.deak@intel.com> (v1)

Refer to the GGTT VM consistently as "ggtt->base" instead of just "ggtt",
"vm" or indirectly through other variables like "dev_priv->ggtt.base"
to avoid confusion with the i915_ggtt object itself and PPGTT VMs.

Refer to the GGTT as "ggtt" instead of indirectly through chaining.

As a bonus gets rid of the long-standing i915_obj_to_ggtt vs.
i915_gem_obj_to_ggtt conflict, due to removal of i915_obj_to_ggtt!

v2:
- Added some more after grepping sources with Chris

v3:
- Refer to GGTT VM through ggtt->base consistently instead of ggtt_vm
  (Chris)

v4:
- Convert all dev_priv->ggtt->foo accesses to ggtt->foo.

v5:
- Make patch checker happy

Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Having provided for_each_engine_id() for cases where the third (id)
argument is useful, we can now replace all the remaining instances with
a simpler version that takes only two parameters. In many cases, this
also allows the elimination of the local variable used in the iterator
(usually 'i').

v2:
    s/dev_priv/(dev_priv__)/ in body of for_each_engine_masked() [Chris Wilson]
Signed-off-by: NDave Gordon <david.s.gordon@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1458757194-17783-2-git-send-email-david.s.gordon@intel.com

Refer to Global GTT consistently as GGTT, thus rename dev_priv->gtt
to dev_priv->ggtt and struct i915_gtt to struct i915_ggtt.

Fix a couple of whitespace problems while at it.

v2:
- Fix a typo in commit message.
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>

Split out the part initing the clock gating hooks and move it earlier.
Add a new NOP hook for platforms without the need to apply clockgating
or workaround settings, so that the hook can be called unconditionally.
Also add a WARN for future platforms that forget to add a hook.

The rest of the hooks in intel_init_pm() should be inited in the same
way, but atm some of the hooks are set only conditionally, so before
doing this we need to make the setup unconditional and use instead some
flags.

v2:
- add a NOP hook and WARN if no hook is set for the platform (Chris)
- use the term hook instead of callback for these functions (Jani)
v3:
- remove the GEN4() check it's already covered by earlier platform
  checks (Chris)

CC: Jani Nikula <jani.nikula@intel.com>
CC: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Link: http://patchwork.freedesktop.org/patch/msgid/1458128348-15730-6-git-send-email-imre.deak@intel.com

Some trivial ones, first pass done with Coccinelle:

@@
@@
(
- I915_NUM_RINGS
+ I915_NUM_ENGINES
|
- intel_ring_flag
+ intel_engine_flag
|
- for_each_ring
+ for_each_engine
|
- i915_gem_request_get_ring
+ i915_gem_request_get_engine
|
- intel_ring_idle
+ intel_engine_idle
|
- i915_gem_reset_ring_status
+ i915_gem_reset_engine_status
|
- i915_gem_reset_ring_cleanup
+ i915_gem_reset_engine_cleanup
|
- init_ring_lists
+ init_engine_lists
)

But that didn't fully work so I cleaned it up with:

for f in *.[hc]; do sed -i -e s/I915_NUM_RINGS/I915_NUM_ENGINES/ $f; done
for f in *.[hc]; do sed -i -e s/i915_gem_request_get_ring/i915_gem_request_get_engine/ $f; done
for f in *.[hc]; do sed -i -e s/intel_ring_flag/intel_engine_flag/ $f; done
for f in *.[hc]; do sed -i -e s/intel_ring_idle/intel_engine_idle/ $f; done
for f in *.[hc]; do sed -i -e s/init_ring_lists/init_engine_lists/ $f; done
for f in *.[hc]; do sed -i -e s/i915_gem_reset_ring_cleanup/i915_gem_reset_engine_cleanup/ $f; done
for f in *.[hc]; do sed -i -e s/i915_gem_reset_ring_status/i915_gem_reset_engine_status/ $f; done

v2: Rebase.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

below and a couple manual fixups.

@@
identifier I, J;
@@
struct I {
...
- struct intel_engine_cs *J;
+ struct intel_engine_cs *engine;
...
}
@@
identifier I, J;
@@
struct I {
...
- struct intel_engine_cs J;
+ struct intel_engine_cs engine;
...
}
@@
struct drm_i915_private *d;
@@
(
- d->ring
+ d->engine
)
@@
struct i915_execbuffer_params *p;
@@
(
- p->ring
+ p->engine
)
@@
struct intel_ringbuffer *r;
@@
(
- r->ring
+ r->engine
)
@@
struct drm_i915_gem_request *req;
@@
(
- req->ring
+ req->engine
)

v2: Script missed the tracepoint code - fixed up by hand.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Done by the Coccinelle script below plus a manual
intervention to GEN8_RING_SEMAPHORE_INIT.

@@
expression E;
@@
- struct intel_engine_cs *ring = E;
+ struct intel_engine_cs *engine = E;
<+...
- ring
+ engine
...+>
@@
@@
- struct intel_engine_cs *ring;
+ struct intel_engine_cs *engine;
<+...
- ring
+ engine
...+>
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

The raw watermark values are needed when planes are not part of the state,
but this introduced a regression and possibly an overflow when merging
the watermarks because invalid values may end up used. Solve this by calculating
raw watermarks for all levels, and only setting non-zero values when the level
is valid.

Fixes the SNB warning:
   WARNING: CPU: 1 PID: 25405 at drivers/gpu/drm/i915/intel_pm.c:2580 ilk_program_watermarks+0x7b2/0x9d0 [i915]()
   WARN_ON(wm_lp != 1)
   Modules linked in: i915 drm_kms_helper drm bluetooth fuse iTCO_wdt iTCO_vendor_support syscopyarea sysfillrect sysimgblt fb_sys_fops tpm_tis mei_me e1000e snd_hda_codec_hdmi pcspkr tpm mei i2c_i801 lpc_ich snd_hda_codec snd_hda_core
   CPU: 1 PID: 25405 Comm: kms_universal_p Tainted: G     U  W       4.5.0-rc6apollolake+ #462
   Hardware name:                  /DH67GD, BIOS BLH6710H.86A.0160.2012.1204.1156 12/04/2012
    0000000000000000 ffff88009d42b918 ffffffff8143cfab ffff88009d42b960
    ffffffffa0363580 ffff88009d42b950 ffffffff81082746 ffff8800b9a24928
    ffff88009d42ba00 ffff88009d4a0000 0000000000000000 ffff88009d42ba6c
   Call Trace:
    [<ffffffff8143cfab>] dump_stack+0x4d/0x72
    [<ffffffff81082746>] warn_slowpath_common+0x86/0xc0
    [<ffffffff810827cc>] warn_slowpath_fmt+0x4c/0x50
    [<ffffffffa0292862>] ilk_program_watermarks+0x7b2/0x9d0 [i915]
    [<ffffffffa0292cb7>] ilk_initial_watermarks+0x107/0x120 [i915]
    [<ffffffffa02feffa>] intel_pre_plane_update+0x12a/0x190 [i915]
    [<ffffffffa02ffb36>] intel_atomic_commit+0x546/0xd50 [i915]
    [<ffffffffa012c9e7>] drm_atomic_commit+0x37/0x60 [drm]
    [<ffffffffa0217361>] drm_atomic_helper_disable_plane+0xb1/0xf0 [drm_kms_helper]
    [<ffffffffa011cdb4>] __setplane_internal+0x184/0x280 [drm]
    [<ffffffffa012b57a>] ? drm_modeset_lock_all_ctx+0x9a/0xb0 [drm]
    [<ffffffffa012010f>] drm_mode_setplane+0x13f/0x1c0 [drm]
    [<ffffffffa0111b52>] drm_ioctl+0x142/0x590 [drm]
    [<ffffffffa011ffd0>] ? drm_plane_check_pixel_format+0x50/0x50 [drm]
    [<ffffffff811f2744>] ? mntput+0x24/0x40
    [<ffffffff811d28d4>] ? __fput+0x194/0x200
    [<ffffffffa012dec3>] drm_compat_ioctl+0x33/0x40 [drm]
    [<ffffffffa029e1c2>] i915_compat_ioctl+0x32/0x40 [i915]
    [<ffffffff81228d72>] compat_SyS_ioctl+0xc2/0x330
    [<ffffffff810021d5>] ? exit_to_usermode_loop+0x95/0xb0
    [<ffffffff81002d2e>] do_fast_syscall_32+0x9e/0x210
    [<ffffffff8197faf2>] entry_SYSENTER_compat+0x52/0x70

Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Testcase: kms_universal_plane
Fixes: d81f04c5 ("drm/i915: Allow preservation of watermarks, v2.")
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/56DEA1FC.8080703@linux.intel.comReviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>

Only planes that are part of the state should be used for recalculating
watermarks. For planes not part of the state the previous patch allows
us to re-use the old values since they're calculated even for levels
that are not actively used.

Changes since v1:
- Remove big if from intel_crtc_atomic_check.
- Remove extra newline.
- Remove memset in ilk_compute_pipe_wm.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456826842-32553-2-git-send-email-maarten.lankhorst@linux.intel.comReviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>

As Paulo has noted we can help bisectability by separating computing
watermarks on a noop in 2 separate commits.

This patch no longer clears the crtc watermark state, but recalculates
it completely. Regardless whether a level is used the full values for
each level are calculated. If a level is invalid wm[level].enable is
unset.

Changes since v1:
- Only call ilk_validate_wm_level when level <= usable_level. (Ville)
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/56D6D09E.5040007@linux.intel.comReviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>

This function returns an int, but when ilk_validate_pipe_wm fails it
returns false, which is 0 (success). As a result invalid watermarks
are applied, while they should have been rejected.

Fix this by returning -EINVAL.

Fixes: ed4a6a7c ("drm/i915: Add two-stage ILK-style watermark programming (v11)")
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456918563-28696-1-git-send-email-maarten.lankhorst@linux.intel.comReviewed-by: NMatt Roper <matthew.d.roper@intel.com>

ae80152d ("drm/i915: Rewrite VLV/CHV
watermark code") removed everything that would have used those vars.
Signed-off-by: NEric Engestrom <eric.engestrom@imgtec.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1456763047-28828-1-git-send-email-eric.engestrom@imgtec.com

In addition to calculating final watermarks, let's also pre-calculate a
set of intermediate watermark values at atomic check time.  These
intermediate watermarks are a combination of the watermarks for the old
state and the new state; they should satisfy the requirements of both
states which means they can be programmed immediately when we commit the
atomic state (without waiting for a vblank).  Once the vblank does
happen, we can then re-program watermarks to the more optimal final
value.

v2: Significant rebasing/rewriting.

v3:
 - Move 'need_postvbl_update' flag to CRTC state (Daniel)
 - Don't forget to check intermediate watermark values for validity
   (Maarten)
 - Don't due async watermark optimization; just do it at the end of the
   atomic transaction, after waiting for vblanks.  We do want it to be
   async eventually, but adding that now will cause more trouble for
   Maarten's in-progress work.  (Maarten)
 - Don't allocate space in crtc_state for intermediate watermarks on
   platforms that don't need it (gen9+).
 - Move WaCxSRDisabledForSpriteScaling:ivb into intel_begin_crtc_commit
   now that ilk_update_wm is gone.

v4:
 - Add a wm_mutex to cover updates to intel_crtc->active and the
   need_postvbl_update flag.  Since we don't have async yet it isn't
   terribly important yet, but might as well add it now.
 - Change interface to program watermarks.  Platforms will now expose
   .initial_watermarks() and .optimize_watermarks() functions to do
   watermark programming.  These should lock wm_mutex, copy the
   appropriate state values into intel_crtc->active, and then call
   the internal program watermarks function.

v5:
 - Skip intermediate watermark calculation/check during initial hardware
   readout since we don't trust the existing HW values (and don't have
   valid values of our own yet).
 - Don't try to call .optimize_watermarks() on platforms that don't have
   atomic watermarks yet.  (Maarten)

v6:
 - Rebase

v7:
 - Further rebase

v8:
 - A few minor indentation and line length fixes

v9:
 - Yet another rebase since Maarten's patches reworked a bunch of the
   code (wm_pre, wm_post, etc.) that this was previously based on.

v10:
 - Move wm_mutex to dev_priv to protect against racing commits against
   disjoint CRTC sets. (Maarten)
 - Drop unnecessary clearing of cstate->wm.need_postvbl_update (Maarten)

v11:
 - Now that we've moved to atomic watermark updates, make sure we call
   the proper function to program watermarks in
   {ironlake,haswell}_crtc_enable(); the failure to do so on the
   previous patch iteration led to us not actually programming the
   watermarks before turning on the CRTC, which was the cause of the
   underruns that the CI system was seeing.
 - Fix inverted logic for determining when to optimize watermarks.  We
   were needlessly optimizing when the intermediate/optimal values were
   the same (harmless), but not actually optimizing when they differed
   (also harmless, but wasteful from a power/bandwidth perspective).

Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1456276813-5689-1-git-send-email-matthew.d.roper@intel.com

The assumption when adding the intel_display_power_is_enabled() checks
was that if it returns success the power can't be turned off afterwards
during the HW access, which is guaranteed by modeset locks. This isn't
always true, so make sure we hold a dedicated reference for the time of
the access.
Spotted-by: NMika Kuoppala <mika.kuoppala@intel.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=93441
CC: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455719489-3008-1-git-send-email-imre.deak@intel.com
(cherry picked from commit 4d800030)
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

The assumption when adding the intel_display_power_is_enabled() checks
was that if it returns success the power can't be turned off afterwards
during the HW access, which is guaranteed by modeset locks. This isn't
always true, so make sure we hold a dedicated reference for the time of
the access.
Spotted-by: NMika Kuoppala <mika.kuoppala@intel.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=93441
CC: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1455719489-3008-1-git-send-email-imre.deak@intel.com