Turns out the VLV/CHV system agent doesn't understand memory
latencies, so trying to rely on the PND deadline mechanism is not
going to fly especially when DDR DVFS is enabled. Currently we try to
avoid the problems by lying to the system agent about the deadlines
and setting the FIFO watermarks to 8 cachelines. This however leads to
bad memory self refresh residency.

So in order to satosfy everyone we'll just give up on the deadline
scheme and program the watermarks old school based on the worst case
memory latency.

I've modelled this a bit on the ILK+ approach where we compute multiple
sets of watermarks for each pipe (PM2,PM5,DDR DVFS) and when merge thet
appropriate one later with the watermarks from other pipes. There isn't
too much to merge actually since each pipe has a totally independent
FIFO (well apart from the mess with the partially shared DSPARB
registers), but still decopuling the pipes from each other seems like a
good idea.

Eventually we'll want to perform the watermark update in two phases
around the plane update to avoid underruns due to the single buffered
watermark registers. But that's still in limbo for ILK+ too, so I've not
gone that far yet for VLV/CHV either.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NClint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Read out the current watermark settings from the hardware at driver init
time. This will allow us to compare the newly calculated values against
the currrent ones and potentially avoid needless WM updates.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NClint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Try to update the watermarks on the right side of the plane update. This
is just a temporary hack until we get the proper two part update into
place. However in the meantime this might have some chance of at least
working.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NClint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We want cxsr exit to happen ASAP, so toss in some POSTING_READ()s to
make sure things are really kicked off.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NClint Taylor <Clinton.A.Taylor@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We can't elide the fb tracking invalidate if the buffer is already in
the right domain since that would lead to missed screen updates. I'm
pretty sure I've written this already before but must have gotten lost
unfortunately :(

v2: Chris observed that all internal set_domain users already
correctly do the fb invalidate on their own, hence we can move this
just into the set_domain ioctl instead.

v3: I screwed up setting the invalidate ORIGIN_* correctly (Chris).

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reported-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Tested-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We cannot let IPS enabled with no plane on the pipe:

BSpec: "IPS cannot be enabled until after at least one plane has
been enabled for at least one vertical blank." and "IPS must be
disabled while there is still at least one plane enabled on the
same pipe as IPS." This restriction apply to HSW and BDW.

However a shortcut path on update primary plane function
to make primary plane invisible by setting DSPCTRL to 0
was leting IPS enabled while there was no
other plane enabled on the pipe causing flickerings that we were
believing that it was caused by that other restriction where
ips cannot be used when pixel rate is greater than 95% of cdclok.

v2: Don't mess with Atomic path as pointed out by Ville.

v3: Rebase after a long time and atomic path changes.
    Accept Ville suggestion of not check !fb

v4: Re-factore on dinq

Reference: https://bugs.freedesktop.org/show_bug.cgi?id=85583
Cc: Paulo Zanoni <paulo.r.zanoni@intel.com>
Cc: Jani Nikula <jani.nikula@intel.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Tested-by: NKenneth Graunke <kenneth@whitecape.org>
[danvet: Make it compile]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We can't improve a 0 deviation, so when we find such a divider, skip the
remaining ones they won't be better.

This short-circuit the search for 34 of the 373 test frequencies in the
corresponding i-g-t test (tools/skl_compute_wrpll)

v2: Place the short-circuiting code in skl_compute_wrpll() (Paulo)

(I'm sure nobody will notice the spurious removal of a blank line)
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Suggested-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Broxton is using a different register and different bit ordering
for rps status capabilities.

Also GT perf freqency register is different for Broxton so update
that.
Signed-off-by: NBob Paauwe <bob.j.paauwe@intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently, if an odd divider improves the deviation (minimizes it), we
take that divider. The recommendation is to prefer even dividers.

v2: Move the check at the right place after having inverted the two for
    loops in the previous patch.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The HW validation team came back from further testing with a slightly
changed constraint on the deviation between the DCO frequency and the
central frequency. Instead of +-4%, it's now +1%/-6%.

Unfortunately, the previous algorithm didn't quite cope with these new
constraints, the reason being that it wasn't thorough enough looking at
the possible divider candidates.

The new algorithm looks at all dividers, which is definitely a hammer
approach (we could reduce further the set of dividers to good ones as a
follow up, at the cost of a bit more complicated code). But, at least,
we can now satisfy the +1%/+6% rule for all the "Well known" HDMI
frequencies of my test set (373 entries).

On that subject, the new code is quite extensively tested in
intel-gpu-tools (tools/skl_compute_wrpll).

v2: Fix cycling between central frequencies and dividers (Paulo)
    Properly choose the minimal deviation between postive and negative
    candidates (Paulo).

    On the 373 test frequencies, v2 computes better dividers than v1 (ie
    more even dividers and lower deviation on average):

    v1: average deviation: 206.52
    v2: average deviation: 194.47
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

After Mika's ppgtt cleanup series, all the other free functions have
drm_device as the first parameter, except this one.

No functional changes.
Signed-off-by: NMichel Thierry <michel.thierry@intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

A safer way to update the PDPx registers is sending lri commands, added
in the ring before the batchbuffer start. Otherwise, the ctx must be idle
before trying to change anything (but the ring-tail) in the ctx image. An
example where the ctx won't be idle is lite-restore.

This patch depends on 5b7e4c9c ("drm/i915/gtt: Mark TLBS dirty for gen8+").

v2: Combine lri writes (and save 8 commands). (Mika)
v3: Rebase after ring/req changes, and removed references to deprecated patches.

Cc: Dave Gordon <david.s.gordon@intel.com>
Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NMichel Thierry <michel.thierry@intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

There is no need for atomicity here. Convert all bitmap
operations to nonatomic variants.

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Scratch page is part of struct i915_address_space. Move other
scratch entities into the same struct. This is a preparatory patch
for having only one instance of each scratch_pt/pd.

v2: make commit msg more readable
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: Michel Thierry <michel.thierry@intel.com> (v1)
[danvet: Bikeshed summary to avoid confusion with vmas.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Write page directory entry without using superfluous
indirect function. Also remove unused device parameter
from the encode function.
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Dynamic page table allocation might wake the shrinker
when memory is requested for page table structures.
As this happens when we try to allocate the virtual address
during binding, our vma might be among the targets for eviction.
We should do i915_vma_pin() and do pin early in there like Chris
suggests but this is interim solution.

Shield our vma from shrinker by incrementing pin count before
the virtual address is allocated.

The proper place to fix this would be in gem, inside of
i915_vma_pin(). But we don't have that yet so take the short
cut as a intermediate solution.

Testcase: igt/gem_ctx_thrash
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Lay out scratch page structure in similar manner than other
paging structures. This allows us to use the same tools for
setup and teardown.
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Make paging structure type agnostic *_px macros to access
page dma struct, the backing page and the dma address.

This makes the code less cluttered on internals of
i915_page_dma.

v2: Superfluous const -> nonconst removed
v3: Rebased
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: Michel Thierry <michel.thierry@intel.com> (v2)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

As there is flushing involved when we have done the cpu
write, make functions for mapping for cpu space. Make macros
to map any type of paging structure.

v2: Make it clear tha flushing kunmap is only for ppgtt (Ville)
v3: Flushing fixed (Ville, Michel). Removed superfluous semicolon

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

When we setup page directories and tables, we point the entries
to a to the next level scratch structure. Make this generic
by introducing a fill_page_dma which maps and flushes. We also
need 32 bit variant for legacy gens.

v2: Fix flushes and handle valleyview (Ville)
v3: Now really fix flushes (Michel, Ville)
Reviewed-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This has slipped in somewhere but it was harmless
as we check the page pointer before teardown.
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

All the paging structures are now similar and mapped for
dma. The unmapping is taken care of by common accessors, so
don't overload the reader with such details.

v2: Be consistent with goto labels (Michel)
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

All our paging structures have struct page and dma address
for that page.

Add struct for page/dma address pairs and use it to make
the setup and teardown for different paging structures
identical.

Include the page directory offset also in the struct for legacy
gens. Rename it to clearly point out that it is offset into the
ggtt.

v2: Add comment about ggtt_offset (Michel)
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The legacy mode mm switch and the execlist context assignment
needs dma address for the page directories.

Introduce a function that encapsulates the scratch_pd dma
fallback if no pd is found.

v2: Rebase, s/ring/req
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: Michel Thierry <michel.thierry@intel.com> (v1)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We can have exactly 4GB sized ppgtt with 32bit system.
size_t is inadequate for this.

v2: Convert a lot more places (Daniel)
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Check the allocation area against the known end
of address space instead of against fixed value.

v2: Return ENODEV on internal bugs (Chris)
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

When we touch gen8+ page maps, mark them dirty like we
do with previous gens.

v2: Update comment (Joonas)
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently we don't have any real indication when a pipe gets
enabled/disabled. Add some.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Avoid some 'switch (plane->type)' by storing the fronbuffer_bits in
intel_plane.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
[danvet: use singular frontbuffer_bits in intel_plane since a plan can
only ever have one bit. Discussed with Ville on irc.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The registers and process differ from other platforms. If the hardware
was programmed incorrectly, this will return invalid cdclk values, which
should then cause reprogramming of the hardware.

v2(Matt): Return 19.2 MHz when DE PLL is disabled (Ville)
v3: Make less assumptions about the hardware state (Ville)

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: NBob Paauwe <bob.j.paauwe@intel.com>
Signed-off-by: NMatt Roper <matthew.d.roper@intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Currently object size is returned for the rotated VMA size which can be
bigger than the rotated view itself. Since the binding code pads all
excess size with scratch pages the only minor issue with this is wasting
some GGTT space, but still feels nicer to fix and report the real size.

v2: Rebase for tracking size in bytes instead of pages.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This way data is available as soon as the view is passed into the call chain.

v2: Store size in bytes instead of pages under the appropriate name. (Chris Wilson)

v3: Use uint64_t instead of size_t. (Daniel Vetter)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com> (v2)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It is only used in logging and it doesn't need to exist on its own.

Also it was misleading to log view size as object size.

v2: Improve commit message. (Joonas Lahtinen)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: NJoonas Lahtinen <joonas.lahtinen@linux.intel.com>
[danvet: s/%lu/%zu/ where needed, reported by 0-day.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

With the new DRRS code it kinda sticks out, and we never managed to
get this to work well enough without causing issues. Time to wave
goodbye.

I've decided to keep the logic for programming the reduced clocks
intact, but everything else is gone. If anyone ever wants to resurrect
this we need to redo it all anyway on top of the frontbuffer tracking.
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Acked-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This typo lead to the crtc scaler getting enabled incorrectly and an
evantual state checker mismatch about the scaler_id.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NDamien Lespiau <damien.lespiau@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In Indirect context w/a batch buffer,
WaClearSlmSpaceAtContextSwitch

This WA performs writes to scratch page so it must be valid, this check
is performed before initializing the batch with this WA.

v2: s/PIPE_CONTROL_FLUSH_RO_CACHES/PIPE_CONTROL_FLUSH_L3 (Ville)

v3: GTT bit in scratch address should be mbz (Chris)

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Dave Gordon <david.s.gordon@intel.com>
Signed-off-by: NRafael Barbalho <rafael.barbalho@intel.com>
Signed-off-by: NArun Siluvery <arun.siluvery@linux.intel.com>
Acked-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Must have missed the transition.
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>

The frontbuffer code gives us accurate information about activity,
let's use it. Again this should avoid unecessary updates when multiple
screens are on.

Also realign function paramaters, I couldn't resist that bit of OCD.

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Durgadoss R <durgadoss.r@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>

The current code tracks business across all pipes, but we're only
really interested in the one pipe DRRS is enabled on. Fairly tiny
optimization, but something I noticed while reading the code. But it
might matter a bit when e.g. showing a video or something only on the
external screen, while the panel is kept static.

Also regroup the code slightly: First compute new bitmasks, then take
appropriate actions.

Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Durgadoss R <durgadoss.r@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>

The current code tracks business across all pipes, but we're only
really interested in the one pipe DRRS is enabled on. Fairly tiny
optimization, but something I noticed while reading the code. But it
might matter a bit when e.g. showing a video or something only on the
external screen, while the panel is kept static.

Also regroup the code slightly: First compute new bitmasks, then take
appropriate actions.

Cc: Ramalingam C <ramalingam.c@intel.com>
Cc: Sivakumar Thulasimani <sivakumar.thulasimani@intel.com>
Reviewed-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>