If the timer putting the last forcewake refcount was pending and we
canceled it, we'll leak the corresponding forcewake and RPM references.

v2:
- do the ptr casting at the caller instead of adding a separate helper
  for this (Chris)
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

We do this at runtime and later on now.
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Ville figured out that it needs a full display reset since apparently
a lot more goes down than just the GT. Until that's address it's
better to just diable gpu reset.
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Up until now, contexts had one (and only one) backing object that was
used by the hardware to save/restore render ring contexts (via the
MI_SET_CONTEXT command). Other rings did not have or need this, so
our i915_hw_context struct had a 1:1 relationship with a a real HW
context.

With Logical Ring Contexts and Execlists, this is not possible anymore:
all rings need a backing object, and it cannot be reused. To prepare
for that, rename our contexts to the more generic term intel_context.

No functional changes.
Signed-off-by: NOscar Mateo <oscar.mateo@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Apparently we need to disable VCP unit clock gating around media reset
on g4x.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Clear the reset domain after a succesful GPU reset on ilk. We already
do that on gen4, so let's try to be a bit more consistent. And if
ether render or media reset fails, we might use the leftover value
in the register to pinpoint the culprit.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

All the other bits in the GDSR register are read-only, so we don't have
to preserve them when we perform a GPU reset.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We're using the reset domains bits for g4x on ilk. But on ilk those bits
actually shifted by one bit. Fix it up so that we use the correct bits.

We were actually always writing 0x2 to the reset domain bits, which
is a reserved value. In practice it looks like the hardware ignores that
value since nothing happens if I write that value when there's a 3D
workload running. Writing the _correct_ render domain value actually
makes the GPU stop.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Acked-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We should be waiting for the reset bit to clear, not remain set.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

There are comments in the gen4-5 reset functions stating that we can't
reset render and media without also doing a display reset. But that's
exactly what the code does, ie. we don't perform a display reset. Drop
the bogus comments.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Media force wake get hangs the machine when the system is booted without
displays attached. The assumption is that (at least some versions of)
the firmware has skipped some initialization in that case.

Empirical evidence suggests we need to reset the media force wake
request register in addition to the render one to avoid hangs.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=75895Reported-by: NImre Deak <imre.deak@intel.com>
Reported-by: NDarren Hart <dvhart@linux.intel.com>
Tested-by: NDarren Hart <dvhart@linux.intel.com>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Cc: stable@vger.kernel.org
Signed-off-by: NJani Nikula <jani.nikula@intel.com>

The same register exists for querying and programming eDRAM AKA eLLC. So
we can simply use it. For now, use all the same defaults as we had
for Haswell, since like Haswell, I have no further details.

I do not actually have a part with eDRAM, so I cannot test this.
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Reviewed-by: NBrad Volkin <bradley.d.volkin@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The reg_read whitelist has a gen bitmask to code the gens we're allowing
the register to be read on. Until now, it was a literal, but we can be
a bit more expressive.

To ease the review, a small test program:

  $ cat bit-range.c
  #include <stdio.h>
  #include <stdint.h>

  #define U32_C(x)		x ## U
  #define GENMASK(h, l)		(((U32_C(1) << ((h) - (l) + 1)) - 1) << (l))
  #define GEN_RANGE(l, h)	GENMASK(h, l)

  int main(int argc, char **argv)
  {
	printf("0x%08x\n", GEN_RANGE(1, 1));
	printf("0x%08x\n", GEN_RANGE(1, 2));
	printf("0x%08x\n", GEN_RANGE(4, 4));
	printf("0x%08x\n", GEN_RANGE(4, 5));
	printf("0x%08x\n", GEN_RANGE(1, 31));
	printf("0x%08x\n", GEN_RANGE(4, 8));

	return 0;
  }
  $ ./bit-range
  0x00000002
  0x00000006
  0x00000010
  0x00000030
  0xfffffffe
  0x000001f0
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

These defines are only used in intel_uncore.c.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

That function isn't used outside this file anymore.
Signed-off-by: NDamien Lespiau <damien.lespiau@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

To avoid WARNs when we call it.

Testcase: igt/pm_pc8/reg-read-ioctl
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=75693Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

So far force_wake_timer was only used by gen6_gt_force_wake_put. Since
we always had balanced gen6_gt_force_wake_get/put calls, we could
guarantee balanced calls to intel_runtime_pm_get/put.

Commit 8232644c, "drm/i915: Convert
the forcewake worker into a timer func" started scheduling the
force_wake_timer at gen6_read, which resulted in an unbalanced
runtime_pm refcount.

So this commit just reverts to the old behavior until we can find a
proper way to used delayed force_wake from the register read/write
macros without leaving the runtime_pm refcounts unbalanced and without
runtime suspending the driver while forcewake is active.

Testcase: igt/pm_pc8/rte
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=76544Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Without this the new drv_suspend/forcewake subtest I've created
doesn't result in immediately visible failures.

Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This is a regression introduced in

commit 0294ae7b
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date:   Thu Mar 13 12:00:29 2014 +0000

    drm/i915: Consolidate forcewake resetting to a single function

The reordered setup sequence ended up calling del_timer_sync before
the timer was set up correctly, resulting in endless hilarity when
loading the driver.

Compared to Ben's patch (which moved around the setup_timer call to
sanitize_early) this moves the sanitize_early call around in the
driver load call. This way we avoid calling setup_timer again in the
resume code (where we also call sanitize_early).

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Cc: Ben Widawsky <benjamin.widawsky@intel.com>
Tested-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=76242Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We have two paths that try to reset the forcewake registers back to
known good values, with slightly different semantics and levels of
paranoia. Combine the two by passing a parameter to either restore the
forcewake status or to clear our bookkeeping, and raise the paranoia
level to max.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Now that we regularly defer the forcewake dance to a timer func, it is
likely to fire after we disable the device during suspend. This
generates an oops as we detect inconsistency in the hardware state. So
before suspend, we want to complete the outstanding dance and generally
sanitize the registers before handing back to the BIOS.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In the upcoming patches we'll need to access the rest of the fields in
the punit power gating register, so prepare for that.

v2:
- add doc reference for the power well subsystem IDs (Jesse)
- remove IDs for non-existant DPIO_RX[23] subsystems (Jesse)
Signed-off-by: NImre Deak <imre.deak@intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

As we now have intel_uncore_forcewake_reset() no need
to do explicit put after reset.

v2: rebase
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

as they don't exists.

v2: rename gen6_*_mt_* to gen7_*_mt_* as they never get called
    with gen6 (Chris)
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: Ben Widawsky <ben@bwidawsk.net> (v1)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

When we get control from BIOS there might be mt forcewake
bits already set. This causes us to do double mt get
without proper clear/ack sequence.

Fix this by clearing mt forcewake register on init,
like we do with older gens.
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It occured to me that when we're trying to wake up both render
and media wells on VLV, we might end up calling the low level
force_wake_get/put two times even though one call would be
enough. Make that happen by figuring out which wells really
need to be woken up based on the forcewake counts.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDeepak S <deepak.s@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

VLV is the only platform where we increment/decrement the forcewake
count around register access. Drop the inc/dec on VLV to make the
forcewake code a bit more unified.

The inc/dec are not necessary since we hold the uncore lock around
the whole operation.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDeepak S <deepak.s@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Use the render/media specific forcewake counts to properly restore the
forcewake status after a GPU reset on VLV.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NDeepak S <deepak.s@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

I could swear this was already happening in the current code...

Also, put the reads and writes in a generic place, so we don't forget
it again when we add runtime PM support to new platforms.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Just to be sure...
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Because we shouldn't be runtime suspended when forcewake is supposed
to be enabled.
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NImre Deak <imre.deak@intel.com>
[danvet: Update commit message - no WARN expected since the bugfix for
issues hit with this assert is already in. And resolve conflicts with
the change from worker to timer for the delayed fw release.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

When we call gen6_gt_force_wake_put we don't actually put force_wake,
we just schedule gen6_force_wake_work through mod_delayed_work, and
that will eventually release force_wake.

The problem is that we call intel_runtime_pm_put directly at
gen6_gt_force_wake_put, so most of the times we put our runtime PM
reference before the delayed work happens, so we may runtime suspend
while force_wake is still supposed to be enabled if the graphics
autosuspend_delay_ms is too small.

Now the nice thing about the current code is that after it triggers
the delayed work function it gets a refcount, and it only triggers the
delayed work function if refcount is zero. This guarantees that when
we schedule the funciton, it will run before we try to schedule it
again, which simplifies the problem and allows for the current
solution to work properly (hopefully!).

v2: - Keep the VLV refcounts balanced (Jesse)
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We don't want to suffer scheduling delay when turning off the GPU after
waking it up to touch registers. Ideally, we only want to keep the GPU
awake for the register access sequence, with a single forcewake dance on
the first access and release immediately after the last. We set a timer
on the first access so that we only dance once and on the next scheduler
tick, we drop the forcewake again.

This moves the cleanup routine from the common i915 workqueue to a timer
func so that we don't anger powertop, and drop the forcewake again
quicker.

v2: Enable the deferred force_wake_put for regular register reads as
    well.
v3: Beautification and make sure we disable forcewake when shutting
    down.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Sometimes generic driver code gets forcewake explicitly by
gen6_gt_force_wake_get(), which check forcewake_count before accessing
hardware. However the register access with gen8_write function access
low level hw accessors directly, ignoring the forcewake_count. This
leads to nested forcewake get from hardware, in ring init and possibly
elsewhere, causing forcewake ack clear errors and/or hangs.

Fix this by checking the forcewake count also in gen8_write

v2: Read side doesn't care about shadowed registers,
    Remove __needs_put funkiness from gen8_write. (Ville)
    Improved commit message.

References: https://bugs.freedesktop.org/show_bug.cgi?id=74007Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Cc: Ben Widawsky <benjamin.widawsky@intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NMika Kuoppala <mika.kuoppala@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Nothing's changed here; we just need to bump the generation check.
Signed-off-by: NKenneth Graunke <kenneth@whitecape.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

We need to have the address space when reserving space for the objects.
Since the address space and context are tied together, and reserve
occurs before context switch (for good reason), we must lookup our
context earlier in the process.

This leaves some room for optimizations where we no longer need to use
ctx_id in certain places. This will be addressed in a subsequent patch.

Important tricky bit:
Because slow relocations during execbuffer drop struct_mutex

Perhaps it would be best to acquire the reference when we get the
context, but I'll save that for another day (note I have written the
patch before, and I found the changes required to be uglier than this).

Note that since we currently access everything via context id, and not
the data structure this is fine, though not desirable. The next change
attempts to get the context only once via the context ID idr lookup, and
as such, the following can happen:

CTX-A is created, refcount = 1
CTX-A execbuf, mutex dropped
close IOCTL called on CTX-A, refcount = 0
CTX-A resumes in execbuf.

v2: Rebased on top of
commit b6359918
Author: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Date:   Wed Oct 30 15:44:16 2013 +0200

    drm/i915: add i915_get_reset_stats_ioctl

v3: Rebased on top of
commit 25b3dfc8
Author: Mika Westerberg <mika.westerberg@linux.intel.com>
Date:   Tue Nov 12 11:57:30 2013 +0200

Author: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Date:   Tue Nov 26 16:14:33 2013 +0200

    drm/i915: check context reset stats before relocations
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Not all registers need forcewake even if they're not shadowed.
Add the missing check to gen8_writeX() to avoid needless forcewake
usage when writing eg. display registers.

v2: Use straight up <0x40000 check instead of NEEDS_FORCE_WAKE()
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The BIOS or someone else might have done something bad and there
might be old GT FIFO erros reported in GTFIFODBG. Clear those out
in intel_uncore_early_sanitize() to make sure we don't mistake them
for our problems.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

These are needed when we cat the debugfs and sysfs files.

V2: - Rebase
V3: - Rebase
V4: - Rebase
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This patch adds the initial infrastructure to allow a Runtime PM
implementation that sets the device to its D3 state. The patch just
adds the necessary callbacks and the initial infrastructure.

We still don't have any platform that actually uses this
infrastructure, we still don't call get/put in all the places we need
to, and we don't have any function to save/restore the state of the
registers. This is not a problem since no platform uses the code added
by this patch. We have a few people simultaneously working on runtime
PM, so this initial code could help everybody make their plans.

V2: - Move some functions to intel_pm.c
    - Remove useless pm_runtime_allow() call at init
    - Remove useless pm_runtime_mark_last_busy() call at get
    - Use pm_runtime_get_sync() instead of 2 calls
    - Add a WARN to check if we're really awake

V3: - Rebase.

V4: - Don't need to call pci_{save,restore}_state and
      pci_set_power_sate, since they're already called by the PCI
      layer
    - Remove wrong pm_runtime_enable() call at init_runtime_pm
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>