Document the non-VM functions in radeon_gart.c

v2: adjust per Christian's suggestions
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Adds documentation to most of the functions in
radeon_ring.c

v2: adjust per Christian's suggestions
v3: adjust per Christian's latest patches
v4: adjust per my latest changes
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Adds documentation to most of the functions in
radeon_fence.c

v2: address Christian's comments:
- split common concept description into it's own comment
- fix description of intr parameter
- Improve description of -EDEADLK error
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Adds documentation to most of the functions in
radeon_asic.c
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Adds documentation to most of the functions in
radeon_irq_kms.c
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Adds documentation to most of the functions in
radeon_kms.c
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Adds documentation to most of the functions in
radeon_device.c

v2: split out general descriptions as per Christian's
comments.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Add support for using memory buffers rather than
scratch registers.  Some rings may not be able to
write to scratch registers.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Just store the index in the ring structure.
Idea taken from one of Jerome's wip rptr patches.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

When submitting a CONST_IB, emit a SWITCH_BUFFER
packet before the CONST_IB.  This isn't strictly necessary
(the driver will work fine without it), but is good practice
and allows for more flexible DE/CE sychronization options
in the future.  Current userspace drivers do not take
advantage of the CE yet.

v2: - clean up code flow a bit
    - no need to flush caches for CONST IB
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Const IBs are executed on the CE not the CP, so we can't
fence them in the normal way.

So submit them directly before the IB instead, just as
the documentation says.

v2: keep the extra documentation
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Otherwise we can encounter out of memory situations under extreme load.

v2: add documentation for the new function
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Otherwise the sa managers out of memory
handling doesn't work.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Try to save whatever is on the rings when
we encounter an lockup.

v2: Fix spelling error. Free saved ring data if reset fails.
    Add documentation for the new functions.
v3: Some more spelling fixes
v4: It doesn't make sense to save anything if all fences
    are signaled
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Before emitting any indirect buffer, emit the offset of the next
valid ring content if any. This allow code that want to resume
ring to resume ring right after ib that caused GPU lockup.

v2: use scratch registers instead of storing it into memory
v3: skip over the surface sync for ni and si as well
v4: use SET_CONFIG_REG instead of PACKET0
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Making it easier to control when it is executed.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Just restore the page table instead. Addressing three
problem with this change:

1. Calling vm_manager_suspend in the suspend path is
   problematic cause it wants to wait for the VM use
   to end, which in case of a lockup never happens.

2. In case of a locked up memory controller
   unbinding the VM seems to make it even more
   unstable, creating an unrecoverable lockup
   in the end.

3. If we want to backup/restore the leftover ring
   content we must not unbind VMs in between.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Just reinitialize the shader content on resume instead.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

The IB pool is in gart memory, so it is completely
superfluous to unpin / repin it on suspend / resume.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

It's not critical, but the current code isn't
100% correct.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

For a normal suspend/resume we allready wait for
the rings to be empty, and for a suspend/reasume
in case of a lockup we REALLY don't want to wait
for anything.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Start with last signaled fence number instead
of last emitted one.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

It is possible that radeon_fence_process is called
after writeback is disabled for suspend, leading
to an invalid read of register 0x0.

This fixes a problem for me where the fence value
is temporary incremented by 0x100000000 on
suspend/resume.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

We don't need to pad anything if the number of dwords
written to the ring already matches the requirements.

Fixes some "writting more dword to ring than expected"
warnings.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

GPU reset need to be exclusive, one happening at a time. For this
add a rw semaphore so that any path that trigger GPU activities
have to take the semaphore as a reader thus allowing concurency.

The GPU reset path take the semaphore as a writer ensuring that
no concurrent reset take place.

v2: init rw semaphore
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Don't return success if scheduling the IB fails, otherwise
we end up with an oops in ttm_eu_fence_buffer_objects.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Waiting for a fence can fail for different reasons,
the most common is a deadlock.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of returning the error handle it directly
and while at it fix the comments about the ring lock.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

It was stuck right in the middle of the gart functions.
Move next to the bm_disable function and where it is used.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Consolidate the CS functions to one section of the file.
Previously they were spread all around.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Try to remove or replace the cs_mutex with a
vm_mutex where it is still needed.

v2: fix locking order
v3: rebased on drm-next
Signed-off-by: NChristian König <deathsimple@vodafone.de>

So we can skip the locking. Also renames sw_int to
ring_int, cause that better matches its purpose.
Signed-off-by: NChristian Koenig <christian.koenig@amd.com>

1. It is really dangerous to have more than one
   spinlock protecting the same information.

2. radeon_irq_set sometimes wasn't called with lock
   protection, so it can happen that more than one
   CPU would tamper with the irq regs at the same
   time.

3. The pm.gui_idle variable was assuming that the 3D
   engine wasn't becoming idle between testing the
   register and setting the variable. So just remove
   it and test the register directly.

v2: Also handle the hpd irq code the same way.
v3: Rename hpd parameter for clarification.
Signed-off-by: NChristian Koenig <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

The spinlock was actually there to protect the
rptr, but rptr was read outside of the locked area.

Also we don't really need a spinlock here, an
atomic should to quite fine since we only need to
prevent it from being reentrant.

v2: Keep the spinlock....
v3: Back to an atomic again after finding & fixing the real bug.
Signed-off-by: NChristian Koenig <christian.koenig@amd.com>

Signed-off-by: NChristian Koenig <christian.koenig@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

It is a rw_semaphore now and only write locked
while changing the clock. Also the lock is renamed
to better reflect what it is protecting.

v2: Keep the ttm_vm_ops on IGPs
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Move inter ring syncing with semaphores into the
existing ring allocations, with that we need to
lock the ring mutex only once.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

v2: BUG_ON not matching rings.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

It is completely unnecessary to create fences
before they are emitted, so remove it and a bunch
of checks if fences are emitted or not.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>