Stop fiddling with jiffies, always wait for RADEON_FENCE_JIFFIES_TIMEOUT.
Consolidate the two wait sequence implementations into just one function.
Activate all waiters and remember if the reset was already done instead of
trying to reset from only one thread.

v2: clear reset flag earlier to avoid timeout in IB test
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Otherwise just reinitialize from scratch on resume,
and so make it more likely to succeed.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Changing the UVD BOs offset on suspend/resume doesn't work because the VCPU
internally keeps pointers to it. Just keep it always pinned and save the
content manually.

Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=66425

v2: fix compiler warning
v3: fix CIK support

Note: a version of this patch needs to go to stable.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

UVD ring can't use scratch thus it does need writeback buffer to keep
a valid address or radeon_ring_backup will trigger a kernel fault.

It's ok to not unpin the write back buffer on suspend as it leave in
gtt and thus does not need eviction.

v2: Fix the uvd case.

Reported and tracked by Wojtek <wojtask9@wp.pl>
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Also init the scratch reg to zero on the UVD ring.
This fixes UVD on AGP based cards.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Just everything needed to decode videos using UVD.

v6: just all the bugfixes and support for R7xx-SI merged in one patch
v7: UVD_CGC_GATE is a write only register, lockup detection fix
v8: split out VRAM fallback changes, remove support for RV770,
    add support for HEMLOCK, add buffer sizes checks
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

radeon_fence_wait_empty_locked should not trigger GPU reset as no
place where it's call from would benefit from such thing and it
actually lead to a kernel deadlock in case the reset is triggered
from pm codepath. Instead force ring completion in place where it
makes sense or return early in others.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Force all fence to signal if GPU reset failed so no process get stuck
on waiting fence.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

The dma ring can't write to register thus have to write to memory
its fence value. This ensure that it doesn't try to use scratch
register for dma ring fence driver.

Should fix:
https://bugs.freedesktop.org/show_bug.cgi?id=58166Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Convert #include "..." to #include <path/...> in drivers/gpu/.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
Acked-by: NDave Airlie <airlied@redhat.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: NDave Jones <davej@redhat.com>

Remove redundant DRM UAPI header #inclusions from drivers/gpu/.

Remove redundant #inclusions of core DRM UAPI headers (drm.h, drm_mode.h and
drm_sarea.h).  They are now #included via drmP.h and drm_crtc.h via a preceding
patch.

Without this patch and the patch to make include the UAPI headers from the core
headers, after the UAPI split, the DRM C sources cannot find these UAPI headers
because the DRM code relies on specific -I flags to make #include "..."  work
on headers in include/drm/ - but that does not work after the UAPI split without
adding more -I flags.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
Acked-by: NDave Airlie <airlied@redhat.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: NDave Jones <davej@redhat.com>

Let's allow GCC to optimize better.

This exposed some five unused functions, but this patch doesn't remove them.
Signed-off-by: NLauri Kasanen <cand@gmx.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Only increase the higher 32bits if we really detect a wrap around.

v2: instead of increasing the higher 32bits just use the higher
    32bits from the last emitted fence.
v3: also use last emitted fence value as upper limit.

The intention of this patch is to make fences as robust as
they where before introducing 64bit fences. This is
necessary because on older systems it looks like the fence
value gets corrupted on initialization.

Fixes:
https://bugs.freedesktop.org/show_bug.cgi?id=51344

Should also fix:
https://bugs.freedesktop.org/show_bug.cgi?id=54129
https://bugs.freedesktop.org/show_bug.cgi?id=54662
https://bugzilla.redhat.com/show_bug.cgi?id=846505
https://bugzilla.redhat.com/show_bug.cgi?id=845639

3.5 needs a separate patch due to changes in the
fence code.  Will send that out separately.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Adds documentation to most of the functions in
radeon_fence.c

v2: address Christian's comments:
- split common concept description into it's own comment
- fix description of intr parameter
- Improve description of -EDEADLK error
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Otherwise the sa managers out of memory
handling doesn't work.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Start with last signaled fence number instead
of last emitted one.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

It is possible that radeon_fence_process is called
after writeback is disabled for suspend, leading
to an invalid read of register 0x0.

This fixes a problem for me where the fence value
is temporary incremented by 0x100000000 on
suspend/resume.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of returning the error handle it directly
and while at it fix the comments about the ring lock.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

v2: BUG_ON not matching rings.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

It is completely unnecessary to create fences
before they are emitted, so remove it and a bunch
of checks if fences are emitted or not.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

/ssd/git/drm-core-next/drivers/gpu/drm/radeon/radeon_fence.c: In function ‘radeon_debugfs_fence_info’:
/ssd/git/drm-core-next/drivers/gpu/drm/radeon/radeon_fence.c:606:7: warning: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 3 has type ‘long long int’ [-Wformat]
Signed-off-by: NDave Airlie <airlied@redhat.com>

It never really belonged there in the first place.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Directly use the suballocator to get small chunks of memory.
It's equally fast and doesn't crash when we encounter a GPU reset.

v2: rebased on new SA interface.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Use one wait queue for all rings. When one ring progress, other
likely does to and we are not expecting to have a lot of waiter
anyway.

Also add a fence_wait_any that will wait until the first fence
in the fence array (one fence per ring) is signaled. This allow
to wait on all rings.

v2: some minor cleanups and improvements.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Some callers illegal called fence_wait_next/empty
while holding the ring emission mutex. So don't
relock the mutex in that cases, and move the actual
locking into the fence code.

v2: Don't try to unlock the mutex if it isn't locked.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Using 64bits fence sequence we can directly compare sequence
number to know if a fence is signaled or not. Thus the fence
list became useless, so does the fence lock that mainly
protected the fence list.

Things like ring.ready are no longer behind a lock, this should
be ok as ring.ready is initialized once and will only change
when facing lockup. Worst case is that we return an -EBUSY just
after a successfull GPU reset, or we go into wait state instead
of returning -EBUSY (thus delaying reporting -EBUSY to fence
wait caller).

v2: Remove left over comment, force using writeback on cayman and
    newer, thus not having to suffer from possibly scratch reg
    exhaustion
v3: Rebase on top of change to uint64 fence patch
v4: Change DCE5 test to force write back on cayman and newer but
    also any APU such as PALM or SUMO family
v5: Rebase on top of new uint64 fence patch
v6: Just break if seq doesn't change any more. Use radeon_fence
    prefix for all function names. Even if it's now highly optimized,
    try avoiding polling to often.
v7: We should never poll the last_seq from the hardware without
    waking the sleeping threads, otherwise we might lose events.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This convert fence to use uint64_t sequence number intention is
to use the fact that uin64_t is big enough that we don't need to
care about wrap around.

Tested with and without writeback using 0xFFFFF000 as initial
fence sequence and thus allowing to test the wrap around from
32bits to 64bits.

v2: Add comment about possible race btw CPU & GPU, add comment
    stressing that we need 2 dword aligned for R600_WB_EVENT_OFFSET
    Read fence sequenc in reverse order of GPU write them so we
    mitigate the race btw CPU and GPU.

v3: Drop the need for ring to emit the 64bits fence, and just have
    each ring emit the lower 32bits of the fence sequence. We
    handle the wrap over 32bits in fence_process.

v4: Just a small optimization: Don't reread the last_seq value
    if loop restarts, since we already know its value anyway.
    Also start at zero not one for seq value and use pre instead
    of post increment in emmit, otherwise wait_empty will deadlock.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Instead of all this humpy pumpy with recursive
mutex (which also fixes only halve of the problem)
move the actual gpu reset out of the fence code,
return -EDEADLK and then reset the gpu in the
calling ioctl function.

v2: Split removal of radeon_mutex into separate patch.
    Return -EAGAIN if reset is successful.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

It's never used and so practically superfluous.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

As discussed with Michel that name better
describes the behavior of this function.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

We should signal the caller that we haven't waited at all.

v2: only change fence_wait_next not fence_wait_last.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Previusly multiple rings could trigger multiple GPU
resets at the same time.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

It makes no sense at all to have more than one flag.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Different rings have different criteria to test
if they are stuck.

v2: rebased on current drm-next
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Silly bad return path.

Reported-and-Tested-by: Mikko Vinni
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
CC: stable@vger.kernel.org
Signed-off-by: NDave Airlie <airlied@redhat.com>

Use semaphores to sync buffers across rings in the CS
ioctl.  Add a reloc flag to allow userspace to skip
sync for buffers.

agd5f: port to latest CS ioctl changes.

v2: add ring lock/unlock to make sure changes hit the ring.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Add a start fence driver helper function which will be call
once for each ring and will compute cpu/gpu addr for fence
depending on wether to use wb buffer or scratch reg.

This patch replace initialize fence driver separately which
was broken in regard of GPU lockup. The fence list for created,
emited, signaled must be initialize once and only from the
asic init callback not from the startup call back which is
call from the gpu reset.

v2: With this in place we no longer need to know the number of
    rings in fence_driver_init, also writing to the scratch reg
    before knowing its offset is a bad idea.

v3: rebase on top of change to previous patch in the serie
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

That naming seems to make more sense, since we not
only want to run PM4 rings with it.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>