Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For very large page table updates, we can exceed the
size of the ring.  To avoid this, use an IB to perform
the page table update.

v2(ck): cleanup the IB infrastructure and the use it instead
        of filling the struct ourself.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>

fetch the reset mask and check if the relevant ring flags
are set to determine whether the ring is hung or not.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It's better to halt the engines before we disable the MC.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When we attempt the reset the GPU, look at the status registers
to determine what blocks need to be reset.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Update the code to better match the recommended
programming sequence for soft reset.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Used by all asic families from r600+.
Flag for the vbios and later instances of the driver
that the GPU is hung.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

After refactoring the _cs logic, we ended up with many
macros and constants that #define the same thing.
Clean'em up.
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: NMarek Olšák <maraeo@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Fixes a hard lock in the gpu reset code after the
rework for DMA support (0ecebb9e
"drm/radeon: switch to a finer grained reset for evergreen")
due to not bailing before the MC shutdown if the relevant engines
are idle.

Discussion:
http://lists.freedesktop.org/archives/dri-devel/2013-January/032985.htmlReported-by: NEldad Zack <eldad@fogrefinery.com>
Tested-by: NEldad Zack <eldad@fogrefinery.com>
Acked-by: NPaul Menzel <paulepanter@users.sourceforge.net>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

No change in functionality as we currently set all the reset
flags.

v2: fix typo
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This try to reset the dma engine when performing gpu reset. Hopefully
bringing back the gpu dma engine in sane state.

v2: agd5f: fix dma reset on cayman/TN, add support for SI
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

To help debug dma related lockup.

v2: agd5f: update SI as well
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Need to verify for copies involving registers.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Async DMA has a special packet for contiguous pt updates
which saves overhead.

v2: rebase
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Pretty much the same as cayman.  Some changes to the copy
packets.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Redirect invalid memory accesses to the default page
instead of locking up the memory controller. Also
enable the invalid memory access interrupts and
start spamming system log with it.

v2 (agd5f): fix up against 2 level PT changes
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

"Whether" is misspelled in various comments across the tree; this
fixes them. No code changes.
Signed-off-by: NAdam Buchbinder <adam.buchbinder@gmail.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

This register is needed for streamout to work properly.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>

Handle requests that won't fit into a single packet.

v2: pe needs to increase as well.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Otherwise the next IB might start reading commands
with the page table still invalid.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The actual set up and assignment of VM page tables
is done on the fly in radeon_gart.c.

v2: update vm size comments
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Use the new WRITE_DATA packet rather than the legacy
ME_WRITE packet.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Pass the vm and ring index rather than an IB.  This allows
us to use the vm_flush interface for non-IB cases in the
future.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

This is the preferred packet for writing data to memory
or registers on SI.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Convert #include "..." to #include <path/...> in drivers/gpu/.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
Acked-by: NDave Airlie <airlied@redhat.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: NDave Jones <davej@redhat.com>

PDE/PTE update code uses CP ring for memory writes.
All page table entries are preallocated for now in alloc_pt().

It is made as whole because it's hard to divide it to several patches
that compile and doesn't break anything being applied separately.

Tested on cayman card.

v2: rebased on top of "refactor set_page chipset interface v3",
    code cleanups

v3: switched offsets calc macros to inline funcs where possible,
    remove pd_addr from radeon_vm, switched RADEON_BLOCK_SIZE define,
    to 9 (and PTE_COUNT to 1 << BLOCK_SIZE)

v4 (ck): move "incr" documentation to previous patch, cleanup and
         document RADEON_VM_* constants, change commit message to
         our usual format, simplify patch allot by removing
         everything current not necessary, disable SI workaround.

v5: (agd5f): Fix typo in tables_size calculation in
             radeon_vm_alloc_pt().  Second line should have been
             '+=' rather than '='.

v6: fix npdes calculation. In scenario when pfns to be mapped overlap
two PDE spans:

   +-----------+-------------+
   | PDE span  | PDE span    |
   +-----------+----+--------+
          |         |
          +---------+
          | pfns    |
          +---------+

the following npdes calculation gives incorrect result:

npdes = (nptes >> RADEON_VM_BLOCK_SIZE) + 1;

For the case above picture it should give npdes = 2, but gives one.

This patch corrects it by rounding last pfn up to 512 border,
first - down to 512 border and then subtracting and dividing by 512.

v7: Make npde calculation clearer, fix ndw calculation.

v8: (agd5f): reserve enough for 2 full VM PTs, add some
             additional comments.

v9: fix typo in npde calculation
Signed-off-by: NDmitry Cherkasov <Dmitrii.Cherkasov@amd.com>
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Let's allow GCC to optimize better.

This exposed some five unused functions, but this patch doesn't remove them.
Signed-off-by: NLauri Kasanen <cand@gmx.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Move binding onto the ring, simplifying handling a bit.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Move flushing the VMs as function into the rings.
First step to make VM operations async.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

It actually isn't very useful.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Store a reference to the VM into the IB structure, that
makes calculating the IBs address a bit less complicated.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

It was only used for dynpm, but has been replaced with
a better implementation using fences.  Remove it.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Returns a snapshot of the GPU clock counter.  Needed
for certain OpenGL extensions.

v2: agd5f
- address Jerome's comments
- add function documentation
Signed-off-by: NMarek Olšák <maraeo@gmail.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The sixteen bank case wasn't handled here, leading to GPU
crashes because of userspace miscalculation.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Cc: stable@vger.kernel.org
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add support for using memory buffers rather than
scratch registers.  Some rings may not be able to
write to scratch registers.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

When submitting a CONST_IB, emit a SWITCH_BUFFER
packet before the CONST_IB.  This isn't strictly necessary
(the driver will work fine without it), but is good practice
and allows for more flexible DE/CE sychronization options
in the future.  Current userspace drivers do not take
advantage of the CE yet.

v2: - clean up code flow a bit
    - no need to flush caches for CONST IB
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Before emitting any indirect buffer, emit the offset of the next
valid ring content if any. This allow code that want to resume
ring to resume ring right after ib that caused GPU lockup.

v2: use scratch registers instead of storing it into memory
v3: skip over the surface sync for ni and si as well
v4: use SET_CONFIG_REG instead of PACKET0
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Making it easier to control when it is executed.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Just restore the page table instead. Addressing three
problem with this change:

1. Calling vm_manager_suspend in the suspend path is
   problematic cause it wants to wait for the VM use
   to end, which in case of a lockup never happens.

2. In case of a locked up memory controller
   unbinding the VM seems to make it even more
   unstable, creating an unrecoverable lockup
   in the end.

3. If we want to backup/restore the leftover ring
   content we must not unbind VMs in between.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>