GTT caching was disabled by default on gen8 due to not working with
big pages. Some information suggests that it got fixed, but still
GTT caching has been left disabled by default. Or could be it just
meant that the default was changed to off, and hence the problem
got solved.

Enable GTT caching in the hopes of some performance increase.
Whether or not the big pages issue has been fixed is irrelevant
at this stage since we don't use big pages.

This gives me a 1-2% improvement in xonotic on my BSW. Haven't tried
BDW, but supposedly it has larger TLBs so might not benefit as much.
On HSW GTT caching is enabled by default.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

GEN8_L3SQCREG1 isn't saved in the context (verified by going through
a context dump), and so we shouldn't be using the ring w/a code to
initialize it. Also Bspec explicitly talks about MMIO and writing it
with the CPU.

Additionally there's another w/a WaTempDisableDOPClkGating:bdw which
tells us to disable DOP clock gating around the GEN8_L3SQCREG1 write
to make sure everyone notices the change. So let's do that as well.

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We're not using ilk_init_lp_watermarks() on BDW for some reason.
Probably due to the BDW patches and the relevant WM patches landing
roughlly at the same time. Fix it up.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Bspec says we should disable the FDI RX/TX before disabling the PCH
ports. Do so.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Follow the BSpec sequence for the CRT port as well on PCH platforms,
ie. disable the pipe before the port.

Didn't bother looking at DDI in detail yet, so leave that one be even
though the CRT is a PCH port there.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

While at it also remove the redundant/unneeded w/a like done for hdmi
already.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
[danvet: Mention that this also removes the unneeded w/a, as suggested
by Jesse.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

BSpec says we should disable all ports after the pipe on PCH
platforms. Do so. Fixes a pipe off timeout on ILK now caused by
the transcoder B workaround.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently the IBX transcoder B workarounds are not working correctly.
Well, the HDMI one seems to be working somewhat, but the DP one is
definitely busted.

After a bit of experimentation it looks like the best way to make this
work is first disable the port on transcoder B, and then re-enable it
transcoder A, and immediately disable it again.

We can also clean up the code by noting that we can't be called without
a valid crtc. And also note that port A on ILK does not need the
workaround, so let's check for that one too.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

On IBX the SDVO/HDMI register write may be masked when enabling the
port, so it may need to written twice. The HDMI code does this, but
the SDVO code does not. Add the workaround to the SDVO code as well.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently we're always enabling enhanced framing on CPT even if the sink
doesn't support it. Fix this up by actaully looking at what the sink
tells us.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Define a TRANS_DP_PIPE_TO_PORT() to make the CPT DP .get_hw_state()
pipe readout neater.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

intel_dp.c is a mess with all the checks for different
platform/PCH variants and ports. Try to clean it up by recognizing
the following facts:
- IVB port A, and CPT port B/C/D are always the special cases
- VLV/CHV don't have port A
- Using the same kind of logic everywhere makes things much easier to
  parse

So let's move the IVB port A and PCH port B/C/D checks to be done first,
and let the other cases fall through, and always check for these things
using the same logic.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

IBX can have problems with the first write to the port register getting
masked when enabling the port. We are trying to apply the workaround
also when disabling the port where it's not needed, and we also try
to apply it for CPT/PPT as well which don't need it. Just kill it.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
[danvet: Resolve conflict with the remove CHV if block.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The IBX 12bpc port enable toggle is only relevant when enabling
the port, not when disabling it. Also this code doesn't actually
toggle anything, and essentially just writes the port register
one extra time. Furthermore CPT/PPT don't need such workarounds
and yet we include them. Just kill it.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We need to re-init the display hardware when going out of suspend. This
includes:

  - Hooking the PCH to the reset logic
  - Restoring CDCDLK
  - Enabling the DDB power

Among those, only the CDCDLK one is a bit tricky. There's some
complexity in that:

  - DPLL0 (which is the source for CDCLK) has two VCOs, each with a set
    of supported frequencies. As eDP also uses DPLL0 for its link rate,
    once DPLL0 is on, we restrict the possible eDP link rates the chosen
    VCO.
  - CDCLK also limits the bandwidth available to push pixels.

So, as a first step, this commit restore what the BIOS set, until I can
do more testing.

In case that's of interest for the reviewer, I've unit tested the
function that derives the decimal frequency field:

  #include <stdio.h>
  #include <stdint.h>
  #include <assert.h>

  #define ARRAY_SIZE(x) (sizeof(x) / sizeof(*(x)))

  static const struct dpll_freq {
          unsigned int freq;
          unsigned int decimal;
  } freqs[] = {
          { .freq = 308570, .decimal = 0b01001100111},
          { .freq = 337500, .decimal = 0b01010100001},
          { .freq = 432000, .decimal = 0b01101011110},
          { .freq = 450000, .decimal = 0b01110000010},
          { .freq = 540000, .decimal = 0b10000110110},
          { .freq = 617140, .decimal = 0b10011010000},
          { .freq = 675000, .decimal = 0b10101000100},
  };

  static void intbits(unsigned int v)
  {
          int i;

          for(i = 10; i >= 0; i--)
                  putchar('0' + ((v >> i) & 1));
  }

  static unsigned int freq_decimal(unsigned int freq /* in kHz */)
  {
          return (freq - 1000) / 500;
  }

  static void test_freq(const struct dpll_freq *entry)
  {
          unsigned int decimal = freq_decimal(entry->freq);

          printf("freq: %d, expected: ", entry->freq);
          intbits(entry->decimal);
          printf(", got: ");
          intbits(decimal);
          putchar('\n');

          assert(decimal == entry->decimal);
  }

  int main(int argc, char **argv)
  {
          int i;

          for (i = 0; i < ARRAY_SIZE(freqs); i++)
                  test_freq(&freqs[i]);

          return 0;
  }

v2:
  - Rebase on top of -nightly
  - Use (freq - 1000) / 500 for the decimal frequency (Ville)
  - Fix setting the enable bit of HSW_NDE_RSTWRN_OPT (Ville)
  - Rename skl_display_{resume,suspend} to skl_{init,uninit}_cdclk to
    be consistent with the BXT code (Ville)
  - Store boot CDCLK in ddi_pll_init (Ville)
  - Merge dev_priv's skl_boot_cdclk into cdclk_freq
  - Use LCPLL_PLL_LOCK instead of (1 << 30) (Ville)
  - Replace various '0' by SKL_DPLL0 to be a bit more explicit that
    we're programming DPLL0
  - Busy poll the PCU before doing the frequency change. It takes about
    3/4 cycles, each separated by 10us, to get the ACK from the CPU
    (Ville)

v3:
  - Restore dev_priv->skl_boot_cdclk, leaving unification with
    dev_priv->cdclk_freq for a later patch (Daniel, Ville)
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

If the client stalls on a congested request, chosen to be 20ms old to
match throttling, allow the client a free RPS boost.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: s/rq/req/]
[danvet: s/0/NULL/ reported by 0-day build]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

If we have clients stalled waiting for requests, ignore the GPU if it
signals that it should downclock due to low load. This helps prevent
the automatic timeout from causing extremely long running batches from
taking even longer.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Now that we have internal clients, rather than faking a whole
drm_i915_file_private just for tracking RPS boosts, create a new struct
intel_rps_client and pass it along when waiting.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: s/rq/req/]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Since we will often pageflip to an active surface, we will often have to
wait for the surface to be written before issuing the flip. Also we are
likely to wait on that surface in plenty of time before the vblank.
Since we have a mechanism for boosting when a flip misses the expected
vblank, curtain the number of times we RPS boost when simply waiting for
mmioflip.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: s/rq/req/]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Ring switches can occur many times per frame, and are often out of
control, causing frequent RPS boosting for no practical benefit. Treat
the sw semaphore synchronisation as a separate client and only allow it
to boost once per busy/idle cycle.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
[danvet: s/rq/req/]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This trims a little overhead from the common case of not needing to
synchronize between rings.

v2: execlists is special and likes to duplicate code.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently, we only track the last request globally across all engines.
This prevents us from issuing concurrent read requests on e.g. the RCS
and BCS engines (or more likely the render and media engines). Without
semaphores, we incur costly stalls as we synchronise between rings -
greatly impacting the current performance of Broadwell versus Haswell in
certain workloads (like video decode). With the introduction of
reference counted requests, it is much easier to track the last request
per ring, as well as the last global write request so that we can
optimise inter-engine read read requests (as well as better optimise
certain CPU waits).

v2: Fix inverted readonly condition for nonblocking waits.
v3: Handle non-continguous engine array after waits
v4: Rebase, tidy, rewrite ring list debugging
v5: Use obj->active as a bitfield, it looks cool
v6: Micro-optimise, mostly involving moving code around
v7: Fix retire-requests-upto for execlists (and multiple rq->ringbuf)
v8: Rebase
v9: Refactor i915_gem_object_sync() to allow the compiler to better
optimise it.

Benchmark: igt/gem_read_read_speed
hsw:gt3e (with semaphores):
Before: Time to read-read 1024k:		275.794µs
After:  Time to read-read 1024k:		123.260µs

hsw:gt3e (w/o semaphores):
Before: Time to read-read 1024k:		230.433µs
After:  Time to read-read 1024k:		124.593µs

bdw-u (w/o semaphores):             Before          After
Time to read-read 1x1:            26.274µs       10.350µs
Time to read-read 128x128:        40.097µs       21.366µs
Time to read-read 256x256:        77.087µs       42.608µs
Time to read-read 512x512:       281.999µs      181.155µs
Time to read-read 1024x1024:    1196.141µs     1118.223µs
Time to read-read 2048x2048:    5639.072µs     5225.837µs
Time to read-read 4096x4096:   22401.662µs    21137.067µs
Time to read-read 8192x8192:   89617.735µs    85637.681µs

Testcase: igt/gem_concurrent_blit (read-read and friends)
Cc: Lionel Landwerlin <lionel.g.landwerlin@linux.intel.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com> [v8]
[danvet: s/\<rq\>/req/g]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The merged seqno->request conversion from John called request
variables req, but some (not all) of Chris' recent patches changed
those to just rq. We've had a lenghty (and inconclusive) discussion on
irc which is the more meaningful name with maybe at most a slight bias
towards req.

Given that the "don't change names without good reason to avoid
conflicts" rule applies, so lets go back to a req everywhere for
consistency. I'll sed any patches for which this will cause conflicts
before applying.

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: John Harrison <John.C.Harrison@Intel.com>
[danvet: s/origina/merged/ as pointed out by Chris - the first
mass-conversion patch was from Chris, the merged one from John.]
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>

v2:
- set the override disable flag too on stepping F0 (mika)
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

On B0 and C0 steppings the workaround enable bit would be overriden by
default, so the overriding must be disabled.

The WA was added in
commit 83a24979
Author: Nick Hoath <nicholas.hoath@intel.com>
Date:   Fri Apr 10 13:12:26 2015 +0100

    drm/i915/bxt: Add WaForceContextSaveRestoreNonCoherent
Spotted-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Our driver compiles clean (nowadays thanks to 0day) but for me, at least,
it would be beneficial if the compiler threw an error rather than a
warning when it found a piece of suspect code. (I use this to
compile-check patch series and want to break on the first compiler error
in order to fix the patch.)

v2: Kick off a new "Debugging" submenu for i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Acked-by: NJani Nikula <jani.nikula@intel.com>
[danvet: Add "DRM i915" to the menu name as requested by Chris.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The macros we use there are the magic ones that can take either dev or
dev_priv. We'd like to move as much as possible towards dev_priv though.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Couldn't let it go!
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently bxt_resume_prepare() is only used in the runtime-resume path.
Add it to the full S3/S4 path as well.

v2: Rebase on top of the vlv_resume_prepare() shuffling around

Cc: Imre Deak <imre.deak@intel.com>
Reviewed-by: Imre Deak <imre.deak@intel.com> (v1)
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Since DRM_ROTATE is counter clockwise (which is compliant with Xrandr),
and HW rotation is clockwise, swapping 90/270 to work as expected from
userspace.

v2: Rebased
Suggested-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NSonika Jindal <sonika.jindal@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Explain why a few fields of the new pipe_config have their values
preserved, while the others are zeroed.
Signed-off-by: NAnder Conselvan de Oliveira <ander.conselvan.de.oliveira@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Also make the WA comment consistent with the rest, where the stepping
info is not shown.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

ARGB8888 is used for cursors on all platforms so we need to allow it
everywhere.

ABGR8888 is currently only honoured:
  - on VLV/CHV in sprite planes
  - on SKL+ for primary and sprite planes
so only allow it for those platforms.

Note that we only support ARGB8888/ABGR8888 on the primary plane for
SKL/BXT because we have in line of sight the pipe bottom color on those
platforms and because the primary plane programming on VLV/CHV doesn't
anything different for those formats today.

v2: Fix the logic to forbid the creation ABGR2101010 fbs (Ville)
v3: Still allow the creation of ARGB8888 fbs now that cursor planes use
    real fb objects (found by PRTS).
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Making lane stagger calculation common for HDMI and DP

v2: Imre's comments addressed
	- Remove lane stagger from bxt_clk_div and make it a local variable in
	ddi_pll_select
Signed-off-by: NVandana Kannan <vandana.kannan@intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

BUN 1: prop_coeff, int_coeff, tdctargetcnt programming updated and tied to
VCO frequencies. Program i_lockthresh in PORT_PLL_9.

VCO calculated based on the formula:
Desired Output = Port bit rate in MHz (DisplayPort HBR2 is 5400 MHz)
Fast Clock = Desired Output / 2
VCO = Fast Clock * P1 * P2

Prop_coeff, int_coeff, and tdctargetcnt modified according to above
calculation.

BUN 2: Port PLLs require additional programming at certain frequencies -
DCO amplitude in PORT_PLL_10

Review comments from Siva which were addressed in the initial version of the
patch.
	- Change PORT_PLL_LOCK_THRESHOLD to PORT_PLL_LOCK_THRESHOLD_MASK
	- Calculate for HDMI
	- Correct values for vco = 5.4
	- return in case of invalid vco range

v2: Imre's review comments addressed
	- change dcoampovr_en to dcoampovr_en_h
	- change PORT_PLL_DCO_AMP_OVR_EN to PORT_PLL_DCO_AMP_OVR_EN_H
	- Correct lane stagger value for 324MHz
	- Make coef common for HDMI and DP
	- remove superfluous comments

v3: Imre's comments addressed
	- Remove Prop_coeff, int_coeff, tdctargetcnt, dcoampovr_en, gain_ctl,
	dcoampovr_en_h from bxt_clk_div and make them local variables.
Signed-off-by: NVandana Kannan <vandana.kannan@intel.com>
Reviewed-by: Sivakumar Thulasimani <sivakumar.thulasimani@intel.com> [v1]
Cc: Sivakumar Thulasimani <sivakumar.thulasimani@intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Be in line with other features that we have.
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Turn

    [drm:intel_dp_print_rates] source rates: 162000,270000,540000,
    [drm:intel_dp_print_rates] sink rates: 162000,270000,
    [drm:intel_dp_print_rates] common rates: 162000,270000,

into

    [drm:intel_dp_print_rates] source rates: 162000, 270000, 540000
    [drm:intel_dp_print_rates] sink rates: 162000, 270000
    [drm:intel_dp_print_rates] common rates: 162000, 270000
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Cc: Imre Deak <imre.deak@intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>