These macros are used only locally, so move them to the .c file.

No functional change.

v2:
- add init power domain to always-on power wells in the following
  - separate - patch (Paulo)
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

These functions are used only by a single call site and are simple
enough to just fold them in.

Note that in later patches the parts folded in here are further
simplified as we'll remove hsw_{disable,enable}_package_c8 and the NULL
check of the power well enable/disable handlers. All this means that at
the end intel_display_power_get/put() becomes more understandable as we
don't need to jump between two functions when reading the code.

No functional change.

v2:
- clarify the rational for the change (Chris)
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We have two names for the same register CHICKEN_PIPESL_1 and
HSW_PIPE_SLICE_CHICKEN_1. Unify it to just one.

Also rename the FBCQ disable bit to resemble the name we've
given to a similar bit on earlier platforms.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

gen7_enable_fbc() may write to some registers which we've already
touched, so use RMW so that we don't undo any previous updates.

Also note that we implemnt WaFbcAsynchFlipDisableFbcQueue:bdw.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Misplaced parens cause us to totally clobber the CHICKEN_PIPESL_1
registers with 0xffffffff. Move the parens to the correct place
to avoid this.

In particular this caused bit 30 of said registers to be set, which
caused the sprite CSC to produce incorrect results.

Cc: stable@vger.kernel.org
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=72220Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Consistency throughout the code base is good and remove some room for
mistakes (as explained in the "drm/i915: Use a pipe variable to cycle
through the pipes" commit)

So, let's replace the for_each_pipe(i) occurences by for_each_pipe(pipe)
when it's reasonable and practical to do so (eg. when there isn't another
pipe variable already).
Suggested-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

According to BSpec we need to always set this magic bit in ring buffer
mode.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Ben and I believe this will be necessary on production hardware.
Signed-off-by: NKenneth Graunke <kenneth@whitecape.org>
[danvet: Shuffle lines to group all ROW_CHICKEN writes and add a
cautious comment that this might not be needed on production hw.]
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

I believe this will be necessary on production hardware.
Signed-off-by: NKenneth Graunke <kenneth@whitecape.org>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
[danvet: Fix whitespace fail spotted by checkpatch. Also add missing
:bdw w/a tag that Ville spotted.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Since the addition of dev_priv->mm.busy, there's no more need for
dev_priv->pc8.gpu_idle, so kill it.

Notice that when you remove gpu_idle, hsw_package_c8_gpu_idle and
hsw_package_c8_gpu_busy become identical to hsw_enable_package_c8 and
hsw_disable_package_c8, so just use them.

Also, when we boot the machine, dev_priv->mm.busy initially considers
the machine as idle. This is opposed to dev_priv->pc8.gpu_idle, which
considered it busy. So dev_priv->pc8.disable_count has to be
initalized to 1 now.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This got lost when we shuffled around our internal branch and
GEN7_FEATURES macro. There were no HW changes to support FBC, so we just
need to set the flag.

v2: Don't allow FBC for any pipe but A on platforms with DDI. (Paulo)

Cc: Daisy Sun <daisy.sun@intel.com>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The power domains framework is internal to the i915 driver, so pass
drm_i915_private instead of drm_device to its functions.

Also remove a dangling intel_set_power_well() declaration.

No functional change.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This reverts commit 116f2b6d.

This optimization causes widespread corruption in games, and even in
glxgears, on my ivb:gt1. The corruption appears like z-fighting of
overlapping polygons in the HiZ buffer.

The observation ties in very closely with the description of the
optimization disabled by default on IVB:

"The Hierarchical Z RAW Stall Optimization allows non-overlapping
polygons in the same 8x4 pixel/sample area to be processed without
stalling waiting for the earlier ones to write to Hierarchical Z
buffer."

No reason is given for why it is disabled by default, usually for such
optimizations it is that it is incomplete. However, there is no
indication whether this a gt1 only issue either. Before considering
reenabling this optimization, I would first suggest reproducing the
corruption in piglit.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=75623Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Chia-I Wu <olv@lunarg.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

V2:  edit the commit message to contain more info
The W/A spreadsheet says this is still required, but the b-spec says
it's not for BYT-T.  So the documentation is not clear.  However,
our experience with the other SKUs of BYT-I/M on Android and Linux
suggests that setting this bit actually causes GPU hang for certain
OGL benchmark applications.

Removing this bit completely resolves the GPU hangs.
Signed-off-by: NSinclair Yeh <sinclair.yeh@intel.com>
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Add a comment next to our WIZ hashing setup to remind people about the
link between WIZ hashing disable bit and PS/WM thread counts.
Suggested-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

BSpec recommends using 8x4 hashing mode when MSAA is used. But in
practice 16x4 seems to have a slight edge in performance (on IVB and
HSW at least). So just use 16x4.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NAntti Koskipää <antti.koskipaa@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

BSpec recommends using 8x4 hashing mode when MSAA is used. But in
practice 16x4 seems to have a slight edge in performance (on IVB and
HSW at least). So just use 16x4.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NAntti Koskipää <antti.koskipaa@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

BSpec recommends using 8x4 hashing mode when MSAA is used. But in
practice 16x4 seems to have a slight edge in performance (on IVB and
HSW at least). So just use 16x4.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NAntti Koskipää <antti.koskipaa@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The need to set all of the mask bits for 3D_CHICKEN3 was required
only for pre-production hardware.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NAntti Koskipää <antti.koskipaa@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Based on the name, the workaround we implement is
WaStripsFansDisableFastClipPerformanceFix. Unfortunately there's no
description in the w/a database, so this is just a guess.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NAntti Koskipää <antti.koskipaa@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

On SNB we set up WaSetupGtModeTdRowDispatch:snb early in
gen6_init_clock_gating(). That sets a bit in the GEN6_GT_MODE register.
However later we go and disable all the bits in the same register. And
then we go on to set some other bit. So apparently we never actually
implemented this workaround since the "disable all bits" part was there
already before the w/a got supposedly implemented.

These are the relevant commits:

 commit 6547fbdb
 Author: Daniel Vetter <daniel.vetter@ffwll.ch>
 Date:   Fri Dec 14 23:38:29 2012 +0100

    drm/i915: Implement WaSetupGtModeTdRowDispatch

 commit f8f2ac9a
 Author: Ben Widawsky <ben@bwidawsk.net>
 Date:   Wed Oct 3 19:34:24 2012 -0700

    drm/i915: Fix GT_MODE default value

So, let's drop the "disable all bits" part, move both writes to
closer proxomity to each other, and name the WIZ hashing bits
appropriately. BSpec is still a bit confused how the bits should
actually be interpreted, but I took the the description for the
high bit since the low bit part only lists values for a single bit.

Also add a comment about our choice of WIZ hashing mode.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NAntti Koskipää <antti.koskipaa@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Only the hardware really access them, so no need to have cpu
gtt access available.

Split out from Chris vma-bind rework.

Note that this is only possible due to the split-up of the mappable
pin flag into PIN_GLOBAL and PIN_MAPPABLE.

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Ben Widawsky <benjamin.widawsky@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Anything more than just one bool parameter is just a pain to read,
symbolic constants are much better.

Split out from Chris' vma-binding rework patch.

v2: Undo the behaviour change in object_pin that Chris spotted.

v3: Split out misplaced hunk to handle set_cache_level errors,
spotted by Jani.

v4: Keep the current over-zealous binding logic in the execbuffer code
working with a quick hack while the overall binding code gets shuffled
around.

v5: Reorder the PIN_ flags for more natural patch splitup.

v6: Pull out the PIN_GLOBAL split-up again.

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Ben Widawsky <benjamin.widawsky@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NDeepak S <deepak.s@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Everything can be overridden by module parameters, so don't confuse the
users that are using them.

We have RC6 turned on for all platforms which support it, but Ironlake,
so the need to explain the situation is no longer pressing.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NDeepak S <deepak.s@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It wasn't ever used by the caller anyway with the exception of what we
show in sysfs.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NDeepak S <deepak.s@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
[danvet: Apply Deepak's suggestion.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

At one time, we though all future platforms would have the deeper RC6
states. As it turned out, they killed it after Ivybridge, and began
using other means to achieve the power savings (the stuff we need to get
to PC7+).

The enable function was left in a weird state of odd corner cases as a
result. Since the future is now, and we also have some insight into
what's currently the future, we have an opportunity to simplify, and
future proof the function.

NOTE: VLV will be addressed in a subsequent patch. This patch was trying
not to change functionality.

NOTE2: All callers sanitize the return value anyway, so this patch is
simply to have the code make a bit more sense.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NDeepak S <deepak.s@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

If we make sure that all the dev_priv->info usages are wrapped by
INTEL_INFO(), we can easily modify the ->info field to be structure and
not a pointer while keeping the const protection in the INTEL_INFO()
macro.

v2: Rebased onto latest drm-nightly
Suggested-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

According to Bspec we need to disable SF pipelined attribute fetch
whenever SF outputs exceed 16 and normal clip mode is used. A quick
glance at Mesa suggests that these conditions could happen. So let's
just always set the magic bit.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NKenneth Graunke <kenneth@whitecape.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

sysfs changes to rps min and max delay were only triggering an update
of the rps interrupt limits if the active delay required an update.
This change ensures that interrupt limits are always updated.

v2: correct compile issue missed on rebase
v3: add igt testcases to signed-off-by section

Testcase: igt/pm_rps/min-max-config-idle
Testcase: igt/pm_rps/min-max-config-loaded
Signed-off-by: NJeff McGee <jeff.mcgee@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

A check of rps/rc6 state after i915_reset determined that the ring
MAX_IDLE registers were returned to their hardware defaults and that
the GEN6_PMIMR register was set to mask all interrupts. This change
restores those values to their pre-reset states by re-initializing
rps/rc6 in i915_reset. A full re-initialization was opted for versus
a targeted set of restore operations for simplicity and maintain-
ability. Note that the re-initialization is not done for Ironlake,
due to a past comment that it causes problems.

Also updated the rps initialization sequence to preserve existing
min/max values in the case of a re-init. We assume the values were
validated upon being set and do not do further range checking. The
debugfs interface for changing min/max was updated with range
checking to ensure this condition (already present in sysfs
interface).

v2: fix rps logging to output hw_max and hw_min, not rps.max_delay
    and rps.min_delay which don't strictly represent hardware limits.
    Add igt testcase to signed-off-by section.

Testcase: igt/pm_rps/reset
Signed-off-by: NJeff McGee <jeff.mcgee@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

When we enter RC6 and GFX Clocks are off, the voltage remains higher
than Vmin. When we try to set the freq to RPn, it might fail since the
Gfx clocks are down. So to fix this in Gfx idle, Bring the GFX clock up
and set the freq to RPn then move GFx down.

v2: remove vlv_update_rps_cur_delay function. Update commit message (Daniel)

v3: Fix the timeout during wait for gfx clock (Jesse)

v4: addressed comments on set freq and punit wait (Ville)

v5: use wait_for while waiting for GFX clk to be up. (Daniel)
    update cur_delay before requesting min_delay. (Ville)

v6: use wait_for while waiting for punit. (Ville)
Signed-off-by: NDeepak S <deepak.s@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Both Bspec and the W/A database state that WaDisablePSDDualDispatchEnable
is only needed for IVB GT1.

The only real confusion here is that the the W/A database also says to
write to the GT2 only register as well, which is strange if the W/A is
only for GT1.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

IVB GT2 has two registers for these things, and both must be written.

To add a bit more confusion both Bspec and the W/A database state that
WaDisablePSDDualDispatchEnable is only needed for IVB GT1, but the W/A
database also says to write even the second GT2 only register. So I
don't really know what the right thing here is.

Note that Bspec disagrees with the w/a database here, but Ville
confirmed (by asking Chris) that on gt1 the 2nd reg doesn't exist.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
[danvet: Add note as requested by Rodrigo.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The optimization helps IVB too.  No piglit regression.
Signed-off-by: NChia-I Wu <olv@lunarg.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The optimization is available on Ivy Bridge and later, and is disabled by
default.  Enabling it helps certain workloads such as GLBenchmark TRex test.

No piglit regression.

v2
 - no need to save the register before suspend as init_clock_gating can
   correctly program it after resume
 - split IVB change to another commit
Signed-off-by: NChia-I Wu <olv@lunarg.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

When current delay is already at max delay, Let's disable the PM UP
THRESHOLD INTRRUPTS, so that we will not get further interrupts until
current delay is less than max delay, Also request for the PM DOWN
THRESHOLD INTRRUPTS to indicate the decrease in clock freq. and
viceversa for PM DOWN THRESHOLD INTRRUPTS.

v2: Use bool variables (Daniel)

v3: Fix Interrupt masking bit (Deepak)

v4: Use existing symbolic constants in i915_reg.h (Daniel)

v5: Add pm interrupt mask after new_delay calculation (Ville)
Signed-off-by: NDeepak S <deepak.s@intel.com>
[danvet: Pass new_delay by value as suggested by Ville. Also appease
checkpatch.]
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We're disabling a boatload of clock gating features on VLV. Maybe these
days we don't need to do that. At least I'm not aware of any workarounds
with this level of paranoia.

This reverts commit 4e8c84a5.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>