It's pretty much all consolidated now that we've killed AGP. We can move
the one outlier, and defines too.

(Kill some unused defines in the process)
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This allows us to map the PTEs WC. I've not done thorough testing or
performance measurements with this patch, but it should be decent.

This is based on a patch from Jesse with the original commit message
> I've only lightly tested this so far, but the corruption seems to be
> gone if I write the GFX_FLSH_CNTL reg after binding an object.  This
> register should control the TLB for the system agent, which is what CPU
> mapped objects will go through.

It has been updated for the new AGP-less code by me, and included with
it is feedback from the original patch.

v2: Updated to reflect paranoia on pte updates/register posting reads.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by [v1]: Jesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2: Accidently removed an ILK case in i9xx_setup (Nicely found by Chris)

CC: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by [v1] : Jesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This bug existed in the old code, but was easier to fix here in the
rework. Unfortunately gen7 doesn't have a nice way to figure out the
size and we must use a lookup table.

As Jesse pointed out, there is some confusion in the docs about these
definitions. We're picking the one which seems more accurate, but we
really aren't certain.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

As a quick hack we make the old intel_gtt structure mutable so we can
fool a bunch of the existing code which depends on elements in that data
structure. We can/should try to remove this in a subsequent patch.

This should preserve the old gtt init behavior which upon writing these
patches seems incorrect. The next patch will fix these things.

The one exception is VLV which doesn't have the preserved flush control
write behavior. Since we want to do that for all GEN6+ stuff, we'll
handle that in a later patch. Mainstream VLV support doesn't actually
exist yet anyway.

v2: Update the comment to remove the "voodoo"
Check that the last pte written matches what we readback

v3: actually kill cache_level_to_agp_type since most of the flags will
disappear in an upcoming patch

v4: v3 was actually not what we wanted (Daniel)
Make the ggtt bind assertions better and stricter (Chris)
Fix some uncaught errors at gtt init (Chris)
Some other random stuff that Chris wanted

v5: check for i==0 in gen6_ggtt_bind_object to shut up gcc (Ben)
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Reviewed-by [v4]: Chris Wilson <chris@chris-wilson.co.uk>
[danvet: Make the cache_level -> agp_flags conversion for pre-gen6 a
tad more robust by mapping everything != CACHE_NONE to the cached agp
flag - we have a 1:1 uncached mapping, but different modes of
cacheable (at least on later generations). Suggested by Chris Wilson.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This has been introduced in "drm/i915: TLB invalidation with
MI_FLUSH_DW requires a post-sync op".
Reported-by: NFengguang Wu <fengguang.wu@intel.com>
Reported-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The BIOS shouldn't be touching this memory across suspend/resume, so
just leave it alone.  This saves us ~6ms on resume on my T420 (retested
with write combined PTEs).

v2: change gtt restore default on pre-gen4 (Chris)
    move needs_gtt_restore flag into dev_priv
v3: make sure we restore GTT on resume from hibernate (Daniel)
    use opregion support as the cutoff for restore from resume (Chris)
v4: use a better check for opregion (Chris)
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
[danvet: Kill the needs_gtt_restore indirection and check directly for
OpRegion. Also explain in a comment what's going on.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This allows the power related code to run independently of the rest of
the pipeline, extending the resume and init time improvements into
userspace, which would otherwise have been blocked on the struct mutex
if we were doing PCU communication.

v2: Also convert the locking for the rps sysfs interface.
Suggested-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org> (v1)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Communicating via the mailbox registers with the PCU can take quite
awhile.  And updating the ring frequency or enabling turbo is not
something that needs to happen synchronously, so take it out of our init
and resume paths to speed things up (~200ms on my T420).

v2: add comment about why we use a work queue (Daniel)
    make sure work queue is idle on suspend (Daniel)
    use a delayed work queue since there's no hurry (Daniel)
v3: make cleanup symmetric and just call cancel work directly (Daniel)
v4: schedule the work using round_jiffies_up to batch work better (Chris)
v5: fix the right schedule_delayed_work call (Chris)

References: https://bugs.freedesktop.org/show_bug.cgi?id=54089Signed-of-by: NJesse Barnes <jbarnes@virtuougseek.org>
[danvet: bikeshed the placement of the new delayed work, move it to
all the other gen6 power mgmt stuff.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The console lock can be contended, so rather than prevent other drivers
after us from being held up, queue the console suspend into the global
work queue that can happen anytime.  I've measured this to take around
200ms on my T420.  Combined with the ring freq/turbo change, we should
save almost 1/2 a second on resume.

v2: use console_trylock() to try to resume the console immediately (Chris)
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
[danvet: move dev_priv->console_resume_work next to the fbdev
pointer.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Pretty astonishing how far apart these two members landed ... Especially since
I've already removed almost 200 lines in between.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Also, move dev_priv->counter there, it's only used in i915_dma.c

And also move the dri1 dungeon at the end of dev_priv where no one
cares about it.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

And give the structs slightly more generic names. I've decided to keep
the short rps/ips prefix, since that's just easier and less churn.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

dev_priv has grown way too big, and grouping memebers into substructs
and moving them out of line helps re-gain some overview.

Unfortunatley I couldn't just call the substruct save and drop the prefix, since
that will make most member names clash with registers #defines. Changes in
i915_drv.h done by hand, everything else changed with
s/\<save\([A-Z]*\)/regfile.save\1/ in vim.
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

So we can write them properly.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NAntti Koskipää <antti.koskipaa@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

"If ENABLED, PIPE_CONTROL command will flush the in flight data  written
out by render engine to Global Observation point on flush done. Also
Requires stall bit ([20] of DW1) set."

So set the stall bit to ensure proper invalidation.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NAntti Koskipää <antti.koskipaa@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

So store into the scratch space of the HWS to make sure the invalidate
occurs.

v2: use GTT address space for store, clean up #defines (Chris)
v3: use correct #define in blt ring flush (Chris)
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NAntti Koskipää <antti.koskipaa@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
References: https://bugs.launchpad.net/ubuntu/+source/xserver-xorg-video-intel/+bug/1063252Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Workaround for dual port PS dispatch on GT1.

v2: pull in register definition & offset handling
v3: use IVB GT1 macro to get the right regs (Ben)
v4: add for VLV too (Ben)
v5: don't read the reg, it's masked so we'll only enable the one extra bit (Chris)
v6: use a _GT2 suffix for the second reg (Chris)
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NAntti Koskipää <antti.koskipaa@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This allows us to get the right vblank interrupt frequency.

v2: pull in register definition
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NAntti Koskipää <antti.koskipaa@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

References: https://bugs.freedesktop.org/show_bug.cgi?id=50250Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NAntti Koskipää <antti.koskipaa@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

v2: use correct register
v3: remove extra hunks, pull in register definitions & offset check directly
v4: add GT1 vs GT2 distinction for IVB portion (Ben)

References: https://bugs.freedesktop.org/show_bug.cgi?id=50233Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NAntti Koskipää <antti.koskipaa@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Needs to be set on every context restore as well, so set it as part of
the initial state so we can save/restore it.  Note this removes the IVB
workaround value from VLV and uses the default value, just adding in the
L3 cache aging disable bit, since the IVB value is wrong for VLV.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NAntti Koskipää <antti.koskipaa@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This covers the "Disable FDI" section from the CRT mode set sequence.
This disables the FDI receiver and also the FDI pll.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This commit makes hsw_fdi_link_train responsible for implementing
everything described in the "Enable and train FDI" section from the
Hawell CRT mode set sequence documentation. We completely rewrite
hsw_fdi_link_train to match the documentation and we also call it in
the right place.

This patch was initially sent as a series of tiny patches fixing every
little problem of the function, but since there were too many patches
fixing the same function it got a little difficult to get the "big
picture" of how the function would be in the end, so here we amended
all the patches into a single big patch fixing the whole function.

Problems we fixed:

  1 - Train Haswell FDI at the right time.

    We need to train the FDI before enabling the pipes and planes, so
    we're moving the call from lpt_pch_enable to haswell_crtc_enable
    directly.

    We are also removing ironlake_fdi_pll_enable since the PLL
    enablement on Haswell is completely different and is also done
    during the link training steps.

  2 - Use the right FDI_RX_CTL register on Haswell

    There is only one PCH transcoder, so it's always _FDI_RXA_CTL.
    Using "pipe" here is wrong.

  3 - Don't rely on DDI_BUF_CTL previous values

    Just set the bits we want, everything else is zero. Also
    POSTING_READ the register before sleeping.

  4 - Program the FDI RX TUSIZE register on hsw_fdi_link_train

    According to the mode set sequence documentation, this is the
    right place. According to the FDI_RX_TUSIZE register description,
    this is the value we should set.

    Also remove the code that sets this register from the old
    location: lpt_pch_enable.

  5 - Properly program FDI_RX_MISC pwrdn lane values on HSW

  6 - Wait only 35us for the FDI link training

    First we wait 30us for the FDI receiver lane calibration, then we
    wait 5us for the FDI auto training time.

  7 - Remove an useless indentation level on hsw_fdi_link_train

    We already "break" when the link training succeeds.

  8 - Disable FDI_RX_ENABLE, not FDI_RX_PLL_ENABLE

    When we fail the training.

  9 - Change Haswell FDI link training error messages

    We shouldn't call DRM_ERROR when still looping through voltage
    levels since this is expected and not really a failure. So in this
    commit we adjust the error path to only DRM_ERROR when we really
    fail after trying everything.

    While at it, replace DRM_DEBUG_DRIVER with DRM_DEBUG_KMS since
    it's what we use everywhere.

  10 - Try each voltage twice at hsw_fdi_link_train

    Now with Daniel Vetter's suggestion to use "/2" instead of ">>1".
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: Applied tiny bikesheds:
- mention in comment that we test each voltage/emphasis level twice
- realing arguments of the only untouched reg write, it spilled over
  the 80 char limit ...]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We had 2 places using X2 and one place using X1.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Now that we no longer pretend to have flexibility in matching any
north display block with any pch, we can ditch this.

v2: Fix the embarassing rebase fail that Paulo Zanoni spotted.
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Found in Bspec vol4h South Display Engine Registers [CPT, PPT],
section "5.3.1  TRANS_CHICKEN_1—Transcoder Chicken Bits 1"

v2: Make it compile.
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We need to set the timing override chicken bit after fdi link training
has completed and before we enable the transcoder. We also have to
clear that bit again after disabling the pch transcoder.

See "Graphics BSpec: vol4g North Display Engine Registers [IVB],
Display Mode Set Sequence" and "Graphics BSpec: vol4h South Display
Engine Registers [CPT, PPT], South Display Engine Transcoder and FDI
Control, Transcoder Debug and DFT, TRANS_CHICKEN_2" bit 31:

"Workaround : Enable the override prior to enabling the transcoder.
Disable the override after disabling the transcoder."

While at it, use the _PIPE macro for the other TRANS_DP register.

v2: Keep the w/a as-is, but kill the original (but wrongly placed)
workaround introduced in

commit 3bcf603f
Author: Jesse Barnes <jbarnes@virtuousgeek.org>
Date:   Wed Jul 27 11:51:40 2011 -0700

    drm/i915: apply timing generator bug workaround on CPT and PPT

and

commit d4270e57
Author: Jesse Barnes <jbarnes@virtuousgeek.org>
Date:   Tue Oct 11 10:43:02 2011 -0700

    drm/i915: export a CPT mode set verification function

Note that this old code has unconditionally set the w/a, which might
explain why fdi link training sometimes silently fails, and especially
why the auto-train did not seem to work properly.

v3: Paulo Zanoni pointed out that this workaround is also required on
the LPT PCH. And Arthur Ranyan confirmed that this workaround is
requierd for all ports on the pch, not just DP: The important part
is that the bit is set whenever the pch transcoder is enabled, and
that it is _not_ set while the fdi link is trained. It is also
important that the pch transcoder is fully disabled, i.e. we have to
wait for bit 30 to clear before clearing the w/a bit.

Hence move to workaround into enable/disable_transcoder, where the pch
transcoder gets enabled/disabled.

v4: Whitespace changes dropped.

v5: Don't run the w/a on IBX, we only need it on CPT/PPT and LPT.

v6:
- resolve conflicts with Paulo's big hsw vga rework
- s/!IBX/CPT since hsw paths are now all separate, and Paulo's patch
  to implement the equivalent w/a for LPT is already merged.

Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Cc: Paulo Zanoni <przanoni@gmail.com>
Cc: Arthur Ranyan <arthur.j.runyan@intel.com>
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com> (v5)
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org> (v5)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

They are all written for a specific north disaplay->pch combination.
So stop pretending otherwise.
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We don't really support fancy north display/pch combinations, so
put a big yelling WARN_ON in there. It /should/ be impossible, but
alas, the rumours don't stop (mostly due to really early silicon
sometimes using older PCHs).

v2: Fixup the logic fumble noticed by Paulo Zanoni. I should actually
try to test run the patch next time around ...
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This essentially reverts

commit cb0953d7
Author: Adam Jackson <ajax@redhat.com>
Date:   Fri Jul 16 14:46:29 2010 -0400

    drm/i915: Initialize LVDS and eDP outputs before anything else

simply because it doesn't scale: It misses SDVO and DVO panels,
and now with DDI encoders on haswell this is becoming unmanageable.

Instead we simply sort the connector list after everything is
set up.
Reviewed-by: NAdam Jackson <ajax@redhat.com>
Acked-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Userspace seems to like this, see

commit cb0953d7
Author: Adam Jackson <ajax@redhat.com>
Date:   Fri Jul 16 14:46:29 2010 -0400

    drm/i915: Initialize LVDS and eDP outputs before anything else

    This makes them sort to the front in X, which makes them likely to be
    the primary outputs if you haven't specified a preference in your DE,
    which is likely to be what you want.
Signed-off-by: NAdam Jackson <ajax@redhat.com>
Signed-off-by: NEric Anholt <eric@anholt.net>

Sorting the connector list after the fact is much easier than trying
to be clever with the init sequence.
Acked-by: NDave Airlie <airlied@gmail.com>
Reviewed-by: NAdam Jackson <ajax@redhat.com>
Acked-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

On Haswell/LPT we must disable the PCH transcoder before we disable
the FDI, so don't check for disabled FDI there.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This function is only for the previous gens.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

These workarounds are documented on the CRT mode set sequence.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

... instead of "pipe", which is wrong.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

... instead of PIPECONF_INTERLACE_MASK.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
[danvet: applied the change by hand due to patch reorder.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>