Polling to make sure the current GPU frequency matches the last
requested frequency should not be necessay, and if there's some
throttling involved, the two might not match anyway.

Since we're still seeing this trigger occasionally, and it just
introduces a rather pointless 10 ms delay, it seems like better
to kill it off.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The RPS register writing routines use the current value of min/max to
set certain limits and interrupt gating. If we set those afterwards, we
risk setting up the hw incorrectly and losing power management events,
and worse, trigger some internal assertions.

Reorder the calling sequences to be correct, and remove the then
unrequired clamping from inside set_rps(). And for a bonus, fix the bug
of calling gen6_set_rps() from Valleyview.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
CC: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

For DDR data rate reporting by Punit in PUNIT_GPU_FREQ_STS, the actual
data encoding is 00b=800, 01b=1066, 10b=1333, 11b=1333.

Some premium VLV sku will get the DDR_DATA_RATE set as 11.  As a result,
the turbo frequency reporting will be incorrect without this workaround.
Signed-off-by: NChon Ming Lee <chon.ming.lee@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We'll be looking at more than just mem_freq from dev_priv, so
just pass the whole thing.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We're currently miscalculating the VLV graphics clock a little bit.
This is caused by rounding the step to integer MHz, which does not
match reality. Change the formula to match the GUnit HAS to give
more accurate answers.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Either the docs were wrong or the values have changed since the old days
before we had wheels.
Reported-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We don't want it delayed with the RPS work.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In

commit 6efdf354
Author: Imre Deak <imre.deak@intel.com>
Date:   Wed Oct 16 17:25:52 2013 +0300

the check for i915_disable_power_well flag was removed by overlook,
so add it back now.
Reported-by: NPaulo Zanoni <paulo.zanoni@intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Similarly rename the other related functions in the power domain
interface.

Higher level driver code calling these functions knows only about power
domains, not the underlying power wells which may be different on
different platforms. Also these functions really init/cleanup/resume
power domains and only through that all related power wells, so rename
them accordingly.

Note that I left i915_{request,release}_power_well as is, since that
really changes the state only of a single power well (and is HSW
specific). It should also get a better name once we make it more
generic by controlling things through a new audio power domain.

v4:
- use intel prefix instead of i915 everywhere (Paulo)
- use a $prefix_$block_$action format (Daniel)
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The only real need for this field was in
i915_{request,release}_power_well, but there we can get at it by a
container_of magic. Also since in the future we'll have multiple power
wells each with its own power_well struct it makes sense to remove the
field from there where it'd be just redundancy.
Suggested-by: NPaulo Zanoni <paulo.zanoni@intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently we make sure that all power domains are enabled during driver
init and turn off unneded ones only after the first modeset. Similarly
during suspend we enable all power domains, which will remain on through
the following resume until the first modeset.

This logic is supported by intel_set_power_well() in the power domain
framework. It would be nice to simplify the API, so that we only have
get/put functions and make it more explicit on the higher level how this
"power well on during init" logic works. This will make it also easier
if in the future we want to shorten the time the power wells are on.

For this add a new device private flag tracking whether we have the
power wells on because of init/suspend and use only
intel_display_power_get()/put(). As nothing else uses
intel_set_power_well() we can remove it.

This also fixes

commit 6efdf354
Author: Imre Deak <imre.deak@intel.com>
Date:   Wed Oct 16 17:25:52 2013 +0300

    drm/i915: enable only the needed power domains during modeset

where removing intel_set_power_well() resulted in not releasing the
reference on the power well that was taken during init and thus leaving
the power well on all the time. Regression reported by Paulo.

v2:
- move the init_power_on flag to the power_domains struct (Daniel)

v3:
- add note about this being a regression fix too (Paulo)
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In the future we'll need to support multiple power wells, so prepare for
that here. Create a new power domains struct which contains all
power domain/well specific fields. Since we'll have one lock protecting
all power wells, move power_well->lock to the new struct too.

No functional change.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Production HSW does not need it. I confirmed this with Art.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Production IVB does not need it. I confirmed this with Art.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

All our registers which are written through the MCHBAR are defined
descriptively as an offset to the MCHBAR. We had 3 outliers here.
Convert these as well so all registers which are offsets are MCHBAR can
be easily identified/found within the code.

With this, convert DCLK to also follow this format.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

There is no hard need for this to be a spin lock, as we don't take these
locks in irq context from anywhere. An upcoming patch will add calls to
punit read/write functions from within regions protected by this lock
and those functions need a mutex in turn. As a solution for that convert
the spin lock to be a mutex.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It is just cleaner this way and makes it easier to add support for
other HW generations with always-on power wells powering a different
set of domains.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Since we use intel_enable_rc6() now for more than just when we're
enabling RC6, we'll see this message many times, and it is just
confusing.

As an example, calc_residency calls this function whenever poked via
sysfs. This leaves the impression in dmesg that we're constantly
re-enabling RC6.

While at it, move the defines and description from drv.h to intel_pm.c,
since these are only ever used in that code.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Using the 5/6 DDB split make sense only when sprites are enabled.
So check that before we waste any cycles computing the merged
watermarks with the 5/6 DDB split.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Makes the behaviour of the function more clear.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Makes the intention more clear.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This debug print just adds overhead to the watermark merging process,
and doesn't really give enough information to be useful. Just kill
and let's add something much better a bit later.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Fill out the HSW watermark s/w tracking structures with the current
hardware state in intel_modeset_setup_hw_state(). This allows us to skip
the HW state readback during watermark programming and just use the values
we keep around in dev_priv->wm. Reduces the overhead of the watermark
programming quite a bit.

v2: s/init_wm/wm_get_hw_state
    Remove stale comment about sprites
    Make DDB partitioning readout safer
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: Fix whitespace fail.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently hsw_write_vm_values() may write to certain watermark
registers needlessly. For instance if only, say, LP3 changes,
the current code will again disable all LP1+ watermarks even
though only LP3 needs to be reconfigured.

Add an easy to read function that will compute the dirtyness of the
watermarks, and use that information to further optimize the watermark
programming.

v2: Disable LP1+ watermarks around changing LP0 watermarks for Paulo
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

To make it easier to check what watermark updates are actually
necessary, keep copies of the relevant bits that match the current
hardware state.

Also add DDB partitioning into hsw_wm_values as that's another piece
of state we want to track.

We don't read out the hardware state on init yet, so we can't really
start using this yet, but it will be used later.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: Paulo asked for a comment around the memcmp to say that we
depend upon zero-initializing the entire structures due to padding.
But a later patch in this series removes the memcmp again. So this is
ok as-is.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The fbc_wm_enabled member in intel_wm_config is useless for the time
being. The original idea for it was that we'd pre-compute it and so
that the WM merging process could know whether it needs to worry
about FBC watermarks at all.

But we don't have a convenient way to pre-check for the possibility
of FBC being used. intel_update_fbc() should be split up for that.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

On HSW the LP1,LP2,LP3 levels are either 1,2,3 or 1,3,4. We make the
conversion from LPn to to the level at one point current. Later we're
going to do it in a few places, so move it to a separate function.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Using the 5/6 DDB split make sense only when sprites are enabled.
So check that before we waste any cycles computing the merged
watermarks with the 5/6 DDB split.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Move the watermark max computations into haswell_update_wm(). This
allows keeping the 1/2 vs. 5/6 split code in one place, and avoid having
to pass around so many things. We also save a bit of stack space by only
requiring one copy of struct hsw_wm_maximums.

Also move the intel_wm_config out from hsw_compute_wm_parameters() and
pass it it. We'll have some need for it in haswell_update_wm() later.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Let's try to keep using the intermediate intel_pipe_wm representation
for as long as possible. It avoids subtle knowledge about the
internals of the hardware registers when trying to choose the
best watermark configuration.

While at it replace the memset() w/ zero initialization.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

I want to convert hsw_find_best_result() to use intel_pipe_wm, so we
need to move the merging to happen outside hsw_compute_wm_results().
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

No point in re-computing the watermarks for all pipes, when only one
pipe has changed. The watermarks stored under intel_crtc.wm.active are
still valid for the other pipes. We just need to redo the merging.

We can also skip the merge/update procedure completely if the new
watermarks for the affected pipe come out unchanged.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Introduce a new struct intel_pipe_wm which contains all the
watermarks for a single pipe. Use it to unify the LP0 and LP1+
watermark computations so that we can just iterate through the
watermark levels neatly and call ilk_compute_wm_level() for each.

Also add another tool ilk_wm_merge() that merges the LP1+ watermarks
from all pipes. For that, embed one intel_pipe_wm inside intel_crtc that
contains the currently valid watermarks for each pipe.

This is mainly preparatory work for pre-computing the watermarks for
each pipe and merging them at a later time. For now the merging still
happens immediately.

v2: Add some comments about level 0 DDB split and intel_wm_config
    Add WARN_ON for level 0 being disabled
    s/lp_wm/merged
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Needed to prevent display corruption in high res panels.

v2: use correct unit names (Rodrigo)
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Tested-by: NUlrich Drepper <drepper@gmail.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

As we delay the initial RPS enabling (upon boot and after resume), there
is a chance that we may start to render and trigger RPS boosts before we
set up the punit. Any changes we make could result in inconsistent
hardware state, with a danger of causing undefined behaviour. However,
as the boosting is a optional tweak to RPS, we can simply ignore it
whilst RPS is not yet enabled.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Let's try to avoid these confusing negated booleans.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The policy's max frequency is not equal to the CPU's max frequency. The
ring frequency is derived from the CPU frequency, and not the policy
frequency.

One example of how this may differ through sysfs. If the sysfs max
frequency is modified, that will be used for the max ring frequency
calculation.
(/sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq). As far as I
know, no current governor uses anything but max as the default, but in
theory, they could. Similarly distributions might set policy as part of
their init process.

It's ideal to use the real frequency because when we're currently scaled
up on the GPU. In this case we likely want to race to idle, and using a
less than max ring frequency is non-optimal for this situation.

AFAIK, this patch should have no impact on a majority of people.

This behavior hasn't been changed since it was first introduced:
commit 23b2f8bb
Author: Jesse Barnes <jbarnes@virtuousgeek.org>
Date:   Tue Jun 28 13:04:16 2011 -0700

    drm/i915: load a ring frequency scaling table v3

CC: Jesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The intel_flush_primary_plane name actually tells us which plane
we're talking about.

Also reorganize the internals a bit and add a missing POSTING_READ()
to make sure the hardware has seen the changes by the time we
return from the function.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

drm_vblank_init() is too ugly. Make it a bit easier on the eye by
collecting all the per-crtc vblank counters, timestamps etc. to
a structure and just allocate an array of those.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

After applying wait-boost we often find ourselves stuck at higher clocks
than required. The current threshold value requires the GPU to be
continuously and completely idle for 313ms before it is dropped by one
bin. Conversely, we require the GPU to be busy for an average of 90% over
a 84ms period before we upclock. So the current thresholds almost never
downclock the GPU, and respond very slowly to sudden demands for more
power. It is easy to observe that we currently lock into the wrong bin
and both underperform in benchmarks and consume more power than optimal
(just by repeating the task and measuring the different results).

An alternative approach, as discussed in the bspec, is to use a
continuous threshold for upclocking, and an average value for downclocking.
This is good for quickly detecting and reacting to state changes within a
frame, however it fails with the common throttling method of waiting
upon the outstanding frame - at least it is difficult to choose a
threshold that works well at 15,000fps and at 60fps. So continue to use
average busy/idle loads to determine frequency change.

v2: Use 3 power zones to keep frequencies low in steady-state mostly
idle (e.g. scrolling, interactive 2D drawing), and frequencies high
for demanding games. In between those end-states, we use a
fast-reclocking algorithm to converge more quickly on the desired bin.

v3: Bug fixes - make sure we reset adj after switching power zones.

v4: Tune - drop the continuous busy thresholds as it prevents us from
choosing the right frequency for glxgears style swap benchmarks. Instead
the goal is to be able to find the right clocks irrespective of the
wait-boost.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Kenneth Graunke <kenneth@whitecape.org>
Cc: Stéphane Marchesin <stephane.marchesin@gmail.com>
Cc: Owen Taylor <otaylor@redhat.com>
Cc: "Meng, Mengmeng" <mengmeng.meng@intel.com>
Cc: "Zhuang, Lena" <lena.zhuang@intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>