Needed to prevent display corruption in high res panels.

v2: use correct unit names (Rodrigo)
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Tested-by: NUlrich Drepper <drepper@gmail.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Otherwise using any atomic memory operation will lock up the GPU due
to a Haswell hardware bug.

v2: Use the _MASKED_BIT_ENABLE macro.  Drop drm parameter definition.
Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: <stable@vger.kernel.org>
[danvet: Fix checkpatch fail.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

During driver loading we are initializing rps.vlv_work in
valleyview_enable_rps() via the rps.delayed_resume_work delayed work.
This is too late since we are using vlv_work already via
i915_driver_load()->intel_uncore_sanitize()->
intel_disable_gt_powersave(). This at least leads to the following
kernel warning:

 INFO: trying to register non-static key.
 the code is fine but needs lockdep annotation.
 turning off the locking correctness validator.

Fix this by initialzing vlv_work before we call intel_uncore_sanitize().

The regression was introduced in

commit 7dcd2677
Author: Konstantin Khlebnikov <khlebnikov@openvz.org>
Date:   Wed Jul 17 10:22:58 2013 +0400

    drm/i915: fix long-standing SNB regression in power consumption
    after resume

though there was no good reason to initialize the static vlv_work from
another delayed work to begin with (especially since this will happen
multiple times).

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=69397Tested-by: Nshui yangwei <yangweix.shui@intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Cc: stable@vger.kernel.org
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Detangle the additional state of whether or not the hw has the pfit
enabled from whether it has zero size. This allows us to cleanly
distinguish in the code when we expect the pfit to be enabled (for
Haswell pc8), and when the BIOS is confused and needs sanitizing.
Reported-by: Nshui yanwei <yangweix.shui@intel.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=68251Tested-by: Nshui yanwei <yangweix.shui@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We shouldn't disable the trickle feed bits on Haswell. Our
documentation explicitly says the trickle feed bits of PRI_CTL and
CUR_CTL should not be programmed to 1, and the hardware engineer also
asked us to not program the SPR_CTL field to 1. Leaving the bits as 1
could cause underflows.
Reported-by: NArthur Runyan <arthur.j.runyan@intel.com>
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Submitting a batchbuffer which simulates a gpu
hang by doing MI_BATCH_BUFFER_START into itself,
to test hangcheck, started to hard hang the whole box
(IVB). Bisecting lead to this commit:

commit 664b422c2966cd39b8f67e8d53a566ea8c877cd6
Author: Vinit Azad <vinit.azad@intel.com>
Date:   Wed Aug 14 13:34:33 2013 -0700

    drm/i915: Only unmask required PM interrupts

Experimenting with the mask register showed that
unmasking EI UP will prevent the hard hang in IVB and SNB.
HSW doesn't hang with EI UP masked.

Considering we are just disabling interrupts that aren't even
delivered to driver, this change is more likely to paper over some
weirdness in gpu's internal state machine. But until better
explanation can be found, let's trade little bit of power
for stability on these architectures.

v2: - Unmask EI_EXPIRED directly in I915_WRITE (Vinit)
v3: - Only unmask on SNB and IVB

Cc: Vinit Azad <vinit.azad@intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Acked-by: NVinit Azad <vinit.azad@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This patch allows PC8+ states on Haswell. These states can only be
reached when all the display outputs are disabled, and they allow some
more power savings.

The fact that the graphics device is allowing PC8+ doesn't mean that
the machine will actually enter PC8+: all the other devices also need
to allow PC8+.

For now this option is disabled by default. You need i915.allow_pc8=1
if you want it.

This patch adds a big comment inside i915_drv.h explaining how it
works and how it tracks things. Read it.

v2: (this is not really v2, many previous versions were already sent,
     but they had different names)
    - Use the new functions to enable/disable GTIMR and GEN6_PMIMR
    - Rename almost all variables and functions to names suggested by
      Chris
    - More WARNs on the IRQ handling code
    - Also disable PC8 when there's GPU work to do (thanks to Ben for
      the help on this), so apps can run caster
    - Enable PC8 on a delayed work function that is delayed for 5
      seconds. This makes sure we only enable PC8+ if we're really
      idle
    - Make sure we're not in PC8+ when suspending
v3: - WARN if IRQs are disabled on __wait_seqno
    - Replace some DRM_ERRORs with WARNs
    - Fix calls to restore GT and PM interrupts
    - Use intel_mark_busy instead of intel_ring_advance to disable PC8
v4: - Use the force_wake, Luke!
v5: - Remove the "IIR is not zero" WARNs
    - Move the force_wake chunk to its own patch
    - Only restore what's missing from RC6, not everything
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Turns out the BIOS will do this for us as needed, and if we try to do it
again we risk hangs or other bad behavior.

Note that this seems to break libva on ChromeOS after resumes (but
strangely _not_ after booting up).

This essentially reverts

commit b4ae3f22
Author: Jesse Barnes <jbarnes@virtuousgeek.org>
Date:   Thu Jun 14 11:04:48 2012 -0700

    drm/i915: load boot context at driver init time

and

commit b3bf0766
Author: Paulo Zanoni <paulo.r.zanoni@intel.com>
Date:   Tue Nov 20 13:27:44 2012 -0200

    drm/i915: implement WaMbcDriverBootEnable on Haswell
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reported-and-Tested-by: NStéphane Marchesin <marcheu@chromium.org>
[danvet: Add note about impact and regression citation.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Just like we're doing with the other IMR changes.

One of the functional changes is that not every caller was doing the
POSTING_READ.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Un-masking all PM interrupts causes hardware to generate
interrupts regardless of whether the interrupts are enabled
on the DE side. Since turbo only need up/down threshold and
rc6 timeout interrupt, mask all other interrupts bits to avoid
unnecessary overhead/wake up.

Note that our interrupt handler isn't being fired since we do set the
IER bits properly (IIR bits aren't set). The overhead isn't because
our driver is reacting to these interrupts, but because hardware keeps
generating internal messages when PMINTRMSK doesn't mask out the
up/down EI interrupts (which happen periodically).

Change-Id: I6c947df6fd5f60584d39b9e8b8c89faa51a5e827
Signed-off-by: NVinit Azad <vinit.azad@intel.com>
[danvet: Add follow-up explanation of the precise effects from Vinit
as a note to the commit message.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Whenever I need to work with the HSW_PWER_WELL_* register bits I have
to look at the documentation to find out which bit is to request the
power well and which one shows its current state. Rename the bits so I
won't need to look the docs every time.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It's basically the same deal as the RC6+ issues on ivy bridge
except this time with RC6 on sandy bridge. Like last time the
core of the issue is that the timings don't work 100% with our
voltage regulator. So from time to time, the kernel will print
a warning message about the GPU not getting out of RC6. In
particular, I found this fairly easy to reproduce during
suspend/resume.

Changing the threshold to 125000 instead of 50000 seems to fix
the issue. The previous patch used 150000 but as it turns out
this doesn't work everywhere. After getting such a machine, I
bisected the highest value which works, which is 125000, so here
it is.

I also measured the idle power usage before/after this patch and
didn't see a difference on a sandy bridge laptop. On haswell and
up, it makes a big difference, so we want to keep it at 50k
there. It also seems like haswell doesn't have the RC6 issues
that sandy bridge has so the 50k value is fine.
Signed-off-by: NStéphane Marchesin <marcheu@chromium.org>
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Due to a misplaced memset(), we never actually enabled the FBC WM on HSW.
Move the memset() to happen a bit earlier, so that it won't clobber
results->enable_fbc_wm.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The '!' here was not intended.  Since '!' has higher precedence than
compare, it means the check is never true.

This regression was introduced in

commit 71fff20f
Author: Ville Syrjälä <ville.syrjala@linux.intel.com>
Date:   Tue Aug 6 22:24:03 2013 +0300

    drm/i915: Kill fbc_enable from hsw_lp_wm_results
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We're going to want to know the crtc in the watermark code to avoid
doing more work than we have to. We should also pass the plane we're
disabling so that we know where to stick our watermark parameters
without having to go look the plane up.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Give a name to the plane watermark related data we have currently
stored under intel_plane->wm.

We also observe that this data is more or less the same that we have
in the hsw_pipe_wm_parameters structure, so use it there as well.

v2: Make pahole happier
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

There is a bunch of global state that needs to be considered when
checking watermarks for validity. Move most of that to a new
structure intel_wm_config, to avoid having to pass around so
many variables.

One notable thing left out is the DDB partitioning information,
since we often anyway need to check the same watermarks against
both 1/2 and 5/6 DDB partitioning layouts.

v2: s/pipes_active/num_pipes_active
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

There are quite a few variables we need to take into account to
determine the maximum watermark levels, so it feels a bit cleaner
to calculate those rather than just have a bunch of what look like
magic numbers.

v2: s/pipes_active/num_pipes_active
    s/othwewise/otherwise
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Let's call hsw_lp_wm_result intel_wm_level from now on and move it to
i915_drv.h for later use.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Refactor the code a bit to split the watermark level validity check into
a separate function.

Also add hack there that allows us to use it even for LP0 watermarks.
ATM we don't pre-compute/check the LP0 watermarks, so we just have to
clamp them to the maximum and hope things work out.

v2: Add some debug prints when we exceed max WM0
    Kill pointless ret = false' assignment.
    Include the check for the already disabled 'result' which
    got shuffled around when the patchs got reorderd
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We're going to use the 1/2 vs. 5/6 split option already on IVB so the
HSW name is not proper. Just give it an intel_ prefix and move it to
i915_drv.h so that we can use it there later.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We don't need to store the FBC WM enabled status in each watermark
level. We anyway have to reduce it down to a single boolean, so just
delay checking the FBC WM limit until we're computing the final
value.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Refactor the watermarks computation for one level to a separate
function. This function will now set the ->enable flag to true,
even if the watermark level wasn't actually checked yet. In the
future we will delay the checking so we must consider all unchecked
watermarks as possibly valid.

v2: Preserve comment about latency units
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Let's be consistent and always call our variables 'enabled' insted of
the occasional 'enable'.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
[danvet: Spelling fix in the commit message, spotted by Chris.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The DRM layer keeps track of our vblanks and it assumes our vblank
counters only go back to zero when they overflow. The problem is that
when we disable the power well our counters also go to zero, but it
doesn't mean they did overflow. So on this patch we grab the lock and
update last_vblank so the DRM layer won't think our counters
overflowed.

This patch fixes the following intel-gpu-tools test:
./kms_flip --run-subtest blocking-absolute-wf_vblank

Regression introduced by the following commit:

commit bf51d5e2
Author: Paulo Zanoni <paulo.r.zanoni@intel.com>
Date:   Wed Jul 3 17:12:13 2013 -0300
    drm/i915: switch disable_power_well default value to 1

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=66808Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: Added a comment that this might be better done in
drm_vblank_post_modeset in general.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

All the ILK+ WM compute functions take the latency values in 0.1us
units. Add a few comments to remind people about that.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Adjust the current ILK/SNB/IVB watermark codepaths to use the
pre-populated latency values from dev_priv instead of reading
them out from the registers every time.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Return UINT_MAX for the calculated WM level if the latency is zero.
This will lead to marking the WM level as disabled.

I'm not sure if latency==0 should mean that we want to disable the
level. But that's the implication I got from the fact that we don't
even enable the watermark code of the SSKDP register is 0.

v2: Use WARN() to scare people
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Seeing the watermark latency values in dmesg might help sometimes.

v2: Use DRM_ERROR() when expected latency values are missing

Note: We might hit the DRM_ERROR added in this patch and apparently
there's not much we can do about that. But I think it'd be interesting
to figure out whether that actually happens in the real world, so I
didn't apply a s/DRM_ERROR/DRM_DEBUG_KMS/ bikeshed while applying.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: Add note about new error dmesg output.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Rather than pass around the plane latencies, just grab them from
dev_priv nearer to where they're needed. Do the same for cursor
latencies.

v2: Add some comments about latency units
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Rather than having to read the latency values out every time, just
store them in dev_priv.

On ILK and IVB there is a difference between some of the latency
values for different planes, so store the latency values for each
plane type separately, and apply the necesary fixups during init.

v2: Fix some checkpatch complaints
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

ILK has a slightly different way to read out the watermark
latency values. On ILK the LP0 latenciy values are in fact
not stored in any register, and instead we must use fixed
values.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

To verbalize it, one can say, "pin an object into the given address
space." The semantics of pinning remain the same otherwise.

Certain objects will always have to be bound into the global GTT.
Therefore, global GTT is a special case, and keep a special interface
around for it (i915_gem_obj_ggtt_pin).

v2: s/i915_gem_ggtt_pin/i915_gem_obj_ggtt_pin
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

SNB and IVB have slightly a different way to read out the
watermark latency values.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The LP1+ watermark latency values need to be multiplied by 5 to
make them suitable for watermark calculations. However on pre-HSW
platforms we're going to need the raw value later when we have to
write it to the WM_LPn registers' latency field. So delay the
multiplication until it's needed.

Note: Paulo complains that the units of wm (now in 100ns) aren't
really clear and I agree. But that can be fixed later on ...
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: Add a comment about the unit obfuscation.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Move parsing of MCH_SSKPD to a separate function, we'll add other
platforms there later.

Note: Chris spotted an empty struct initializer and wondered whether
that is hiding a compilier warning. Ville explained that it should
have been part of the patch that extends this function to snb/ivb,
which don't have all levels hsw has. I've figured it's ok to keep it
here with a small note.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: Add note about the ominous struct initializer.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The latency values fit in uint16_t, so let's save a few bytes.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The FBC watermark doesn't depend on the latency value, so no point in
passing it in.

Note: It actually depends upon the latency, but only through priv_val
...
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: Add review comment from Paulo to the commit message.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

These functions are appropriate for everything since ILK.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

hsw_wm_get_pixel_rate() isn't specific to HSW. In fact it should be made
to handle all gens, but for now it depends on the PCH panel fitter
state, so give it an ilk_ prefix.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>