The '!' here was not intended.  Since '!' has higher precedence than
compare, it means the check is never true.

This regression was introduced in

commit 71fff20f
Author: Ville Syrjälä <ville.syrjala@linux.intel.com>
Date:   Tue Aug 6 22:24:03 2013 +0300

    drm/i915: Kill fbc_enable from hsw_lp_wm_results
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We're going to want to know the crtc in the watermark code to avoid
doing more work than we have to. We should also pass the plane we're
disabling so that we know where to stick our watermark parameters
without having to go look the plane up.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Give a name to the plane watermark related data we have currently
stored under intel_plane->wm.

We also observe that this data is more or less the same that we have
in the hsw_pipe_wm_parameters structure, so use it there as well.

v2: Make pahole happier
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

There is a bunch of global state that needs to be considered when
checking watermarks for validity. Move most of that to a new
structure intel_wm_config, to avoid having to pass around so
many variables.

One notable thing left out is the DDB partitioning information,
since we often anyway need to check the same watermarks against
both 1/2 and 5/6 DDB partitioning layouts.

v2: s/pipes_active/num_pipes_active
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

There are quite a few variables we need to take into account to
determine the maximum watermark levels, so it feels a bit cleaner
to calculate those rather than just have a bunch of what look like
magic numbers.

v2: s/pipes_active/num_pipes_active
    s/othwewise/otherwise
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Let's call hsw_lp_wm_result intel_wm_level from now on and move it to
i915_drv.h for later use.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Refactor the code a bit to split the watermark level validity check into
a separate function.

Also add hack there that allows us to use it even for LP0 watermarks.
ATM we don't pre-compute/check the LP0 watermarks, so we just have to
clamp them to the maximum and hope things work out.

v2: Add some debug prints when we exceed max WM0
    Kill pointless ret = false' assignment.
    Include the check for the already disabled 'result' which
    got shuffled around when the patchs got reorderd
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We're going to use the 1/2 vs. 5/6 split option already on IVB so the
HSW name is not proper. Just give it an intel_ prefix and move it to
i915_drv.h so that we can use it there later.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We don't need to store the FBC WM enabled status in each watermark
level. We anyway have to reduce it down to a single boolean, so just
delay checking the FBC WM limit until we're computing the final
value.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Refactor the watermarks computation for one level to a separate
function. This function will now set the ->enable flag to true,
even if the watermark level wasn't actually checked yet. In the
future we will delay the checking so we must consider all unchecked
watermarks as possibly valid.

v2: Preserve comment about latency units
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Let's be consistent and always call our variables 'enabled' insted of
the occasional 'enable'.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
[danvet: Spelling fix in the commit message, spotted by Chris.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

All the ILK+ WM compute functions take the latency values in 0.1us
units. Add a few comments to remind people about that.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Adjust the current ILK/SNB/IVB watermark codepaths to use the
pre-populated latency values from dev_priv instead of reading
them out from the registers every time.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Return UINT_MAX for the calculated WM level if the latency is zero.
This will lead to marking the WM level as disabled.

I'm not sure if latency==0 should mean that we want to disable the
level. But that's the implication I got from the fact that we don't
even enable the watermark code of the SSKDP register is 0.

v2: Use WARN() to scare people
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Seeing the watermark latency values in dmesg might help sometimes.

v2: Use DRM_ERROR() when expected latency values are missing

Note: We might hit the DRM_ERROR added in this patch and apparently
there's not much we can do about that. But I think it'd be interesting
to figure out whether that actually happens in the real world, so I
didn't apply a s/DRM_ERROR/DRM_DEBUG_KMS/ bikeshed while applying.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: Add note about new error dmesg output.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Rather than pass around the plane latencies, just grab them from
dev_priv nearer to where they're needed. Do the same for cursor
latencies.

v2: Add some comments about latency units
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Rather than having to read the latency values out every time, just
store them in dev_priv.

On ILK and IVB there is a difference between some of the latency
values for different planes, so store the latency values for each
plane type separately, and apply the necesary fixups during init.

v2: Fix some checkpatch complaints
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

ILK has a slightly different way to read out the watermark
latency values. On ILK the LP0 latenciy values are in fact
not stored in any register, and instead we must use fixed
values.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

To verbalize it, one can say, "pin an object into the given address
space." The semantics of pinning remain the same otherwise.

Certain objects will always have to be bound into the global GTT.
Therefore, global GTT is a special case, and keep a special interface
around for it (i915_gem_obj_ggtt_pin).

v2: s/i915_gem_ggtt_pin/i915_gem_obj_ggtt_pin
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

SNB and IVB have slightly a different way to read out the
watermark latency values.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The LP1+ watermark latency values need to be multiplied by 5 to
make them suitable for watermark calculations. However on pre-HSW
platforms we're going to need the raw value later when we have to
write it to the WM_LPn registers' latency field. So delay the
multiplication until it's needed.

Note: Paulo complains that the units of wm (now in 100ns) aren't
really clear and I agree. But that can be fixed later on ...
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: Add a comment about the unit obfuscation.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Move parsing of MCH_SSKPD to a separate function, we'll add other
platforms there later.

Note: Chris spotted an empty struct initializer and wondered whether
that is hiding a compilier warning. Ville explained that it should
have been part of the patch that extends this function to snb/ivb,
which don't have all levels hsw has. I've figured it's ok to keep it
here with a small note.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: Add note about the ominous struct initializer.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The latency values fit in uint16_t, so let's save a few bytes.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The FBC watermark doesn't depend on the latency value, so no point in
passing it in.

Note: It actually depends upon the latency, but only through priv_val
...
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: Add review comment from Paulo to the commit message.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

These functions are appropriate for everything since ILK.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

hsw_wm_get_pixel_rate() isn't specific to HSW. In fact it should be made
to handle all gens, but for now it depends on the PCH panel fitter
state, so give it an ilk_ prefix.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Don't subtract one from the sprite width before watermark calculations.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

For calculating watermarks we want to know whether sprites are
scaled. Pass that information to update_sprite_watermarks() so that
eventually we may do some watermark pre-computing.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Almost invariably the reason why FBC cannot be turned on is the same
every time (disabled via parameter, too many pipes, pipe too large etc)
as modesetting and framebuffer configuration changes less frequently
than trying to enable FBC.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently, the register access code is split between i915_drv.c and
intel_pm.c. It only bares a superficial resemblance to the reset of the
powermanagement code, so move it all into its own file. This is to ease
further patches to enforce serialised register access.

v2: Scan for random abuse of I915_WRITE_NOTRACE
v3: Take the opportunity to rename the GT functions as uncore. Uncore is
the term used by the hardware design (and bspec) for all functions
outside of the GPU (and CPU) cores in what is also known as the System
Agent.
v4: Rebase onto SNB rc6 fixes
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
[danvet: Wrestle patch into applying and inline
intel_uncore_early_sanitize (plus move the old comment to the new
function). Also keep the _santize postfix for intel_uncore_sanitize.]
[danvet: Squash in fixup spotted by Chris on irc: We need to call
intel_pm_init before intel_uncore_sanitize since the later will call
cancel_work on the delayed rps setup work the former initializes.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The regression fix for gen6+ rps fallout

commit 7dcd2677
Author: Konstantin Khlebnikov <khlebnikov@openvz.org>
Date:   Wed Jul 17 10:22:58 2013 +0400

    drm/i915: fix long-standing SNB regression in power consumption after resume

unintentionally also changed the init sequence ordering between
gt_init and gt_reset - we need to reset BIOS damage like leftover
forcewake references before we run our own code. Otherwise we can get
nasty dmesg noise like

[drm:__gen6_gt_force_wake_mt_get] *ERROR* Timed out waiting for forcewake old ack to clear.

again. Since _reset suggests that we first need to have stuff
initialized (which isn't the case here) call it sanitze instead.

While at it also block out the rps disable introduced by the above
commit on ilk: We don't have any knowledge of ilk rps being broken in
similar ways. And the disable functions uses the default hw state
which is only read out when we're enabling rps. So essentially we've
been writing random grabage into that register.
Reported-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Konstantin Khlebnikov <khlebnikov@openvz.org>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Cc: stable@vger.kernel.org
Tested-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This patch fixes regression in power consumtion of sandy bridge gpu, which
exists since v3.6 Sometimes after resuming from s2ram gpu starts thinking that
it's extremely busy. After that it never reaches rc6 state.

Bug exists since kernel v3.6:

commit b4ae3f22
Author: Jesse Barnes <jbarnes@virtuousgeek.org>
Date:   Thu Jun 14 11:04:48 2012 -0700

    drm/i915: load boot context at driver init time

For some reason RC6 is already enabled at the beginning of resuming process.
Following initliaztion breaks some internal state and confuses RPS engine.
This patch disables RC6 at the beginnig of resume and initialization.

I've rearranged initialization sequence, because intel_disable_gt_powersave()
needs initialized force_wake_get/put and some locks from the dev_priv.

Note: The culprit in the initialization sequence seems to be the write
to MBCTL added in the above mentioned commit. The first version of
this patch just held a forcewake reference across the clock gating
init functions, which seems to have been enought to gather quite a few
positive test reports. But since that smelled a bit like ad-hoc
duct-tape v2 now just disables rps/rc6 across the entire hw setup.

References: https://bugs.freedesktop.org/show_bug.cgi?id=54089
References: https://bugzilla.kernel.org/show_bug.cgi?id=58971
References: https://patchwork.kernel.org/patch/2827634/ (patch v1)
Signed-off-by: NKonstantin Khlebnikov <khlebnikov@openvz.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
[danvet: Add note about v1 vs. v2 of this patch and use standard
layout for the commit citation. Also add the tested-bys from v1 and a
cc: stable.]
Cc: stable@vger.kernel.org (Note: tiny conflict due to the addition of
the backlight lock in 3.11)
Tested-by: Alexander Kaltsas <alexkaltsas@gmail.com> (v1)
Tested-by: rocko <rockorequin@hotmail.com> (v1)
Tested-by: JohnMB <johnmbryant@sky.com> (v1)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

intel_enable_rc6() is used to check if we can compute the RC6 residency
in the sysfs code. Disable this for platforms older than Ironlake.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2: Put the comment a bit closer to the actual write (Paulo Zanoni)
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
[danvet: Fix space before tab.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We also wait for that blank on other platforms but the w/a doesn't
apply there. Not an issue at all.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

At the moment we have the following interrupt enabling sequence:
1. irq preinstall hook
2. enabling the interrupt handler and calling irq postinstall hook
3. enable rps interrupts from the async work

And the folliwing disable sequence:
1. disabling the interrupt handler and calling the uninstall hook
2. disabling the rps interrupt

Since the postinstall hook now always sets up PMIIR, PMIER and PMIMR
to known-good states there no way for an interrupt to sneak in in the
enable sequence, so we can reinstate the WARN lost in

commit eda63ffb
Author: Ben Widawsky <ben@bwidawsk.net>
Date:   Tue May 28 19:22:26 2013 -0700

    drm/i915: Add PM regs to pre/post install

Note that there's some room for future cleanups since most of the
interrupt register clearing in the disable function is rather
redundant. But that's better done in follow-up patches, if at all.

Cc: Ben Widawsky <ben@bwidawsk.net>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The VECS enabling required some changes to how rps interrupts are
enabled/disabled since VECS interrupts are handling with the PM
interrupt registers.

But now that the pre/postinstall sequences is identical for all
platforms with rps support (snb, ivb, hsw, vlv) we can also use the
exact same sequence to actually enable the rps interrupts. Strictly
speaking using spinlocks is overkill on snb/ivb & vlv since they have
no VECS ring, but imo that's more than made up by the common code.

Hence this just unifies the vlv code with the snb-hsw code which
matched exactly before the VECS enabling. See

commit eda63ffb
Author: Ben Widawsky <ben@bwidawsk.net>
Date:   Tue May 28 19:22:26 2013 -0700

    drm/i915: Add PM regs to pre/post install

and

commit 4848405c
Author: Ben Widawsky <ben@bwidawsk.net>
Date:   Tue May 28 19:22:27 2013 -0700

    drm/i915: make PM interrupt writes non-destructive

for why the gen6 code (shared between snb, ivb and hsw) needed to be
changed originally.

v3: Improve the commit message to more clearly spell out why we want
to unify the code and what exactly changes.

Cc: Paulo Zanoni <przanoni@gmail.com>
Cc: Ben Widawsky <ben@bwidawsk.net>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Again extract a common helper. For the postinstall hook things are a
bit more complicated since we have more cases on ilk-hsw/vlv here.

But since vlv was clearly broken by failing to initialize
dev_priv->gt_irq_mask correctly the shared code is clearly justified.

Also kill the PMIER setting in the async rps enable work. I should
have been save, but also clearly looked rather fragile. PMIER setup is
now all down in the irq pre/postinstall hooks.

With this we now have the usual interrupt register sequence for GT/PM
irq registers:

- IER is setup once with all the interrupts we ever need in the
  postinstall hook and never touched again. Exceptions are SDEIER,
  which is touched in the preinstall hook (when the irq handler isn't
  enabled) and then only from the irq handler. And DEIER/VLV_IER with
  is used in the irq handler but also written to once in the
  postinstall hook. But since that write is essentially what enables
  the interrupt and we should always have MSI interrupts we should be
  save. In case we ever have non-MSI interrupts we'd be screwed.

- IIR is cleared in the postinstall hook before we enable/unmask the
  respective interrupt sources. Hence we can't steal an interrupt
  event an accidentally trigger the spurious interrupt logic in the
  core kernel. Note that after some discussion with Ben Widawsky we
  think that we actually should clear the IIR registers in the
  preinstall hook. But doing that is a much larger patch series.

- IMR regs are (usually) all masked off. Those are the only regs
  changed at runtime, which is all protected by dev_priv->irq_lock.

This unification also kills the cargo-culted read-modify-write PM
register setup for VECS. Interrupt setup is done without userspace
being able to interfere, so we better know what values we want to put
into those registers. RMW cycles otoh are really good at papering over
races, until stuff magically blows up and no one has a clue why.

v2: Touch the gen6+ PM interrupt registers only on gen6+.

v3: Improve the commit message to more clearly spell out why we want
to unify the code and what exactly changes.

Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Paulo Zanoni <przanoni@gmail.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
[danvet: Add a comment to explain why the l3 parity interrupt is
special.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Now that the rps interrupt locking isn't clearly separated (at elast
conceptually) from all the other interrupt locking having a different
lock stopped making sense: It protects much more than just the rps
workqueue it started out with. But with the addition of VECS the
separation started to blurr and resulted in some more complex locking
for the ring interrupt refcount.

With this we can (again) unifiy the ringbuffer irq refcounts without
causing a massive confusion, but that's for the next patch.

v2: Explain better why the rps.lock once made sense and why no longer,
requested by Ben.
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>