Inserting additional PTEs has no side-effect for us as the pfn are fixed
for the entire time the object is resident in the global GTT. The
downside is that we pay the entire cost of faulting the object upon the
first hit, for which we in return receive the benefit of removing the
per-page faulting overhead.

On an Ivybridge i7-3720qm with 1600MHz DDR3, with 32 fences,
Upload rate for 2 linear surfaces:	8127MiB/s -> 8134MiB/s
Upload rate for 2 tiled surfaces:	8607MiB/s -> 8625MiB/s
Upload rate for 4 linear surfaces:	8127MiB/s -> 8127MiB/s
Upload rate for 4 tiled surfaces:	8611MiB/s -> 8602MiB/s
Upload rate for 8 linear surfaces:	8114MiB/s -> 8124MiB/s
Upload rate for 8 tiled surfaces:	8601MiB/s -> 8603MiB/s
Upload rate for 16 linear surfaces:	8110MiB/s -> 8123MiB/s
Upload rate for 16 tiled surfaces:	8595MiB/s -> 8606MiB/s
Upload rate for 32 linear surfaces:	8104MiB/s -> 8121MiB/s
Upload rate for 32 tiled surfaces:	8589MiB/s -> 8605MiB/s
Upload rate for 64 linear surfaces:	8107MiB/s -> 8121MiB/s
Upload rate for 64 tiled surfaces:	2013MiB/s -> 3017MiB/s
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: "Goel, Akash" <akash.goel@intel.com>
Testcasee: igt/gem_fence_upload/performance
Reviewed-by: NBrad Volkin <bradley.d.volkin@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Adding missing Display mmio reg offset.

Credits-to: Laws, Philip <philip.laws@intel.com>
Cc: He, Shuang <shuang.he@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Atm, the forcewake refcount will be incorrectly set to zero during
system suspend if there is any reference held via the
i915_forcewake_user debugfs entry.

Fix this by simply not zeroing the sw counters during suspend and
restoring the original state using them. Note that the only other
places where we zeroed the counters were driver load and unload time,
where it was redundant anyway.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78059Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

If the timer putting the last forcewake refcount was pending and we
canceled it, we'll leak the corresponding forcewake and RPM references.

v2:
- do the ptr casting at the caller instead of adding a separate helper
  for this (Chris)
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

pipe_name() returns an ascii character.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Add Broadwell support to i915_frequency_info
and extend i915_max|min_freq_get|set to (gen >= 6).

v2: generalized support for i915_max|min_freq_get|set (Daniel).
Signed-off-by: NTom O'Rourke <Tom.O'Rourke@intel.com>
Reviewed-by: NJeff McGee <jeff.mcgee@intel.com>
[danvet: Fix checkpatch fail.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Certain DVO chips (ns2501 for example) don't like to be accessed unless
the PLL is running. Simply skip the DVO get_hw_state if the DVO port
is disabled.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Using names initializers when filling out the watermark structs
saves you from having go look up the struct definition every
single time.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Now that we have a release hook into i915_gem_object_free, we can move
the explicit call to the internal stolen function and hook it up
throught the callback instead.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Otherwise we incur an unsightly WARNING. The mutex locking is a bit
overkill, but it curbs races and eventially we might grow a locking
check in the vblank wait code to make sure the right crtc lock is
held.

This is fallout from

commit 93937071
Author:     Jesse Barnes <jbarnes@virtuousgeek.org>
AuthorDate: Fri Apr 4 16:12:09 2014 -0700

    drm/i915: warn when a vblank wait times out

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=79612Tested-by: NGuo Jinxian <jinxianx.guo@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
[danvet: Rebase on top of drm core ww locking changes.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

For no reason at all the public docs lack them, and Dave needs them
for his hpd interrupt rework.

Cc: Dave Airlie <airlied@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Replaced ever growing switch for gen version with chained conditionals.
Futre gen's only need to add a new one if they require something different.
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NRobert Beckett <robert.beckett@intel.com>
[danvet: Picked from internal tree and white-wash commit message.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Those LUT where defined in the original sprite patch introducing intel_plane,
but were never used.

  commit b840d907
  Author: Jesse Barnes <jbarnes@virtuousgeek.org>
  Date:   Tue Dec 13 13:19:38 2011 -0800

    drm/i915: add SNB and IVB video sprite support v6
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
[danvet: Pimp commit message as suggested by Damien]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Keeping track of the power domains is a bit messy since crtc->active
is currently updated by the platform hooks, but we need to be aware of
which state transition exactly is going on. Maybe we simply need to
shovel all the power domain handling down into platform code to
simplify this. But doing that requires some more auditing since
currently the ->mode_set callbacks still read some random registers
(to e.g. figure out the reference clocks).

Also note that intel_crtc_update_dpms is always call first/last even
for encoders which have their own dpms functions. Hence we really only
need to update this place here.

Being a quick "does it blow up?" run not really tested yet.

v2: Don't do runtime PM in the DPMS hooks for HAS_DDI platforms since
that is stalled. Also add a comment to explain what's going on.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Working for real this time.  i915_ppgtt_info has all sorts of good stuff
in it and X is running nicely on top.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Conceptually, the MIPI registers are addressed by the MIPI transcoder
index, not the pipe. It doesn't matter right now, because there's a
1:1 relationship between pipes and MIPI transcoders, but that change
allows us to break that link in the future

V1: Created new patch to address Damien's review comment.
Replacing _PIPE calls to _TRANSCODER calls
V2: Re-basing on patch 2
V3: Re-basing on patch 2
V4: Re-basing on patch 2
Signed-off-by: NShashank Sharma <shashank.sharma@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Re-define MIPI register definitions in such a way that most of
the existing DSI code can be re-used for future platforms. Register
definitions are re-written using MMIO offset variable, so that without
changing the existing sequence, same code can be generically applied.

V4: Addressing review comments by Damien and Ville, splitting into two patches
This patch removes all the un-necessary formatting changes from previous patch.
V5: Removed 80 char limit formatting for existing MIPI regs
V6: Removed extra space, change one definition
Signed-off-by: NShashank Sharma <shashank.sharma@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Fixed several double space pointer notations, and added one newline
Signed-off-by: NRobin Schroer <sulamiification@gmail.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Instead of shuffling gfp-masks all the time, use the
shmem_read_mapping_page() helper. Note that __GFP_IO and __GFP_WAIT are
set in mapping_gfp_mask() for i915, so the behavior is still the same.

Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Signed-off-by: NDavid Herrmann <dh.herrmann@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Intel hardware allows the primary plane to be disabled independently of
the CRTC.  Provide custom primary plane handling to allow this.

v8:
 - Pin/unpin properly when clipping causes the primary plane to be
   disabled when it has previously been enabled.
 - s/drm_primary_helper_check_update/drm_plane_helper_check_update/
v7:
 - Clip primary plane to invisible when crtc is disabled since
   intel_crtc->config.pipe_src_{w,h} may be garbage otherwise.
 - Unpin old fb before pinning new one in the "just pin and
   return" case that is used when the crtc is disabled.
 - Don't treat implicit disabling of the primary plane (caused by
   clipping) the same way as explicit disabling (caused by fb=0).
   For implicit disables, we should leave the fb set and pinned,
   whereas for explicit disables we need to unpin the fb before
   primary->fb is cleared.
v6:
 - Pass rectangles to primary helper check function and get plane
   visibility back.
 - Wait for pending pageflips on primary plane update/disable.
 - Allow primary plane to be updated while the crtc is disabled (changes
   will take effect when the crtc is re-enabled if modeset passes -1
   for the fb id).
 - Drop WARN() if we try to disable the primary plane when it's
   already been disabled.  This will happen if the crtc gets disabled
   after the primary plane has already been disabled independently.
v5:
 - Use new drm_primary_helper_check_update() helper function to check
   setplane parameter validity.
 - Swap primary plane's pipe for pre-gen4 FBC (caught by Ville Syrjälä)
 - Cleanup primary plane properly on crtc init failure
v4:
 - Don't add a primary_plane field to intel_crtc; that was left over
   from a much earlier iteration of this patch series, but is no longer
   needed/used now that the DRM core primary plane support has been
   merged.
v3:
 - Provide gen-specific primary plane format lists (suggested by Daniel
   Vetter).
 - If the primary plane is already enabled, go ahead and just call the
   primary plane helper to do the update (suggested by Daniel Vetter).
 - Don't try to disable the primary plane on destruction; the DRM layer
   should have already taken care of this for us.
v2:
 - Unpin fb properly on primary plane disable
 - Provide an Intel-specific set of primary plane formats
 - Additional sanity checks on setplane (in line with the checks
   currently being done by the DRM core primary plane helper)
Reviewed-by: NChon Ming Lee <chon.ming.lee@intel.com>
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In a future patch, we'll allow the primary plane to be disabled by
userspace via the universal plane API.  If a modeset is requested while
the primary plane is disabled, crtc->primary->fb will be NULL which
generally triggers a full modeset (except in fastboot situations).  If
we detect that the crtc is active, but there's no primary plane fb,
we should still allow a simple plane update rather than a full modeset
if the mode isn't actually changing (after re-enabling the primary plane
of course).

v2:
 - Enable plane after set_base to avoid enabling the plane if set_base
   fails, and to make flip+enable atomic (suggested by Ville)
 - Drop BUG to WARN if we somehow enter the 'fb_changed' modeset case
   with the crtc disabled (suggested by Ville)
Reviewed-by: NChon Ming Lee <chon.ming.lee@intel.com>
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

All the date we print is invariant for the lifetime of the driver.
And none of it would be protected by the mode_config.mutex anyway.
So drop it.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This goes all the way back to the introduction of this debugfs file,
even though back then no locking really was required. None of the
intermediate patches fixed this.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This doesn't look possible but a little extra defense against the
improbable is worth it - an oops here could lockup the machine.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Functions that can't fail are such a bliss to work with, it'd be shame
to miss the occasion. The "failure" mode is the DSI connector not being
created, the rest of the initialization can carry on happily.

We weren't even checking that value anyway.
Suggested-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Suggested-by: NShobhit Kumar <shobhit.kumar@intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Shobhit Kumar <shobhit.kumar@intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
[danvet: Also convert the missed return statement due to other patches
merged meanwhile.]
[danvet2: Squash in fixup from Damien to remove empty return; at the
end of intel_dsi_init.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Fix warnings introduced by the following commit -

commit 9c92da2c7c17eea79b6321b37592df0a002d24df
Author: Shobhit Kumar <shobhit.kumar@intel.com>
Date:   Fri May 23 21:35:27 2014 +0530

    drm/i915: Add support for Generic MIPI panel driver

Fixed all except the DRM logging which go beyond line 80
Signed-off-by: NShobhit Kumar <shobhit.kumar@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The DIP registers are a mess on VLV and CHV. The register block on pipe
A is different than the register block on pipes B and C. In order to
handle that using the pipe offsets, we'd need a new pipe offset per
register, which seems wasteful. So instead just use the _PIPE3() macro
to handle these registers.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

These are just single registers so wasting space for the pipe offsets
seems a bit pointless. So just use the _PIPE3() macro instead.

Also rewrite the _PIPE3() macro to be more obvious, and protect the
arguments properly.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
[danvet: Frob conflict.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Now that we forced the clock buffers on in .pre_pll_enable() we
should probably undo the damage after we've turned the PLL off.

We do the clock buffer force enable in the .pre_pll_enable() hook
as we need to know which port is going to be used, but in the disable
case we don't need the port since we just disable the clock buffers
to both channels. So we can do this in chv_disable_pll() instead
of having to add any kind of .post_pll_disable() hook.

v2: Improve the commit message
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Try to force the PHY clock buffer enables to make the clock routing
work.

v2: Fix the pipe B case to actually enable CH0 clock buffers
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

These should make it possible to feed port C from pipe A or port B from
pipe B. Didn't quite seem to work though.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

On CHV, All the freq request should be even. So, we need to make sure we
request the opcode accordingly.

v2: Avoid vairable for freq request (ville)
Signed-off-by: NDeepak S <deepak.s@linux.intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

CHV uses the gen8 shadow register mechanism so we shouldn't be
checking the GT FIFO status.

This effectively removes the posting read, so add an explicit
posting read using FORCEWAKE_ACK_VLV (which is what use in
vlv_forcewake_reset()).
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Skip __gen6_gt_wait_for_thread_c0() on CHV.
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2: Disable media turbo and Add DOWN_IDLE_AVG support (Ville)

v3: Mass rename of the dev_priv->rps variables in upstream.

v4: Rebase against latest code. (Deepak)

v5: Rebase against latest nightly code. (Deepak)

v6: Rename the variables to match the spec (Mika)

v7: change min/max freq variable naming to match spec (Mika)
Signed-off-by: NDeepak S <deepak.s@linux.intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Because of the upcoming vblank interrupt driven watermark update
mechanism we will have use for vblank interrupts during plane
enabling/disabling. So don't call drm_vblank_off() until planes
are off, and call drm_vblank_on() just before we start to enable
the planes.

v2: Pimp commit message (Paulo)
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Support to individually control Media/Render well based on the register access.
Add CHV specific write function to habdle difference between registers
that are sadowed vs those that need forcewake even for writes.

Streamline the CHV forcewake functions just like was done for VLV.

This will also fix a bug in accessing the common well registers,
where we'd end up trying to wake up the wells too many times
since we'd call force_wake_get/put twice per register access, with
FORCEFAKE_ALL both times.

v2: Drop write FIFO for CHV and add comman well forcewake (Ville)
    Re-factor CHV/VLV Forcewake offsets (Ben)

v3: Fix for decrementing fw count in chv read/write. (Deepak)

v4: Squash the patches (Mika)
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
[vsyrjala: Move the register range macros into intel_uncore.c]
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDeepak S <deepak.s@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2: Configure PCBR if BIOS fails allocate pcbr (deepak)

v3: Fix PCBR condition check during CHV RC6 Enable flag set

v4: Fixup PCBR comment msg. (Chris)
    Rebase against latest code (Deak)
    Fixup Spurious hunk (Ben)

v5: Fix PCBR and commentis msg (mika)

v6: Rebase patch on latest nightly (Deepak)
Signed-off-by: NDeepak S <deepak.s@linux.intel.com>
Acked-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

"Because our driver assumes only one panel is PSR capable, and we
already have other PSR information on dev_priv instead of intel_dp. If
we ever support multiple PSR panels, we'll have to move struct
i915_psr to intel_dp anyway." (by Paulo)

v2: Avoid more than one setup. Removing initialization
    and trusting allocation. (By Paulo Zanoni).
v3: rebase.
v4: Adding comment.
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>