Neither complete nor perfect, but solves my problem at hand
and might be useful in the future.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Give the ring functions a separate structure and let the asic
structure point to the ring specific functions. This simplifies
the code and allows us to make changes at only one point.

No change in functionality.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Use the UVD handle information to determine which
which power states to select when using UVD.  For
example, decoding a single SD stream requires much
lower clocks than multiple HD streams.

v2: switch to a cleaner dpm/uvd interface
v3: change the uvd power state while streams
are active if need be
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This commit converts the source of the val_seq counter to
the ww_mutex api. The reservation objects are converted later,
because there is still a lockdep splat in nouveau that has to
resolved first.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

When using UVD, the driver must switch to a special UVD power
state.  In the CS ioctl, switch to the power state and schedule
work to change the power state back, when the work comes up,
check if uvd is still busy and if not, switch back to the user
state, otherwise, reschedule the work.

Note:  We really need some better way to decide when to
switch out of the uvd power state.  Switching power states
while playback is active make uvd angry.

V2: fix locking.

V3: switch from timer to delayed work

V4: check fence driver for UVD jobs, reduce timeout to
    1 second and rearm timeout on activity

v5: rebase on new dpm tree

v6: rebase on interim uvd on demand changes

v7: fix UVD when DPM is disabled

v8: unify non-DPM and DPM UVD handling

v9: remove leftover idle work struct
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <deathsimple@vodafone.de>

On CIK, the compute rings work slightly differently than
on previous asics, however the basic concepts are the same.

The main differences:
- New MEC engines for compute queues
- Multiple queues per MEC:
  - CI/KB: 1 MEC, 4 pipes per MEC, 8 queues per pipe = 32 queues
  -    KV: 2 MEC, 4 pipes per MEC, 8 queues per pipe = 64 queues
- Queues can be allocated and scheduled by another queue
- New doorbell aperture allows you to assign space in the aperture
  for the wptr which allows for userspace access to queues

v2: add wptr shadow, fix eop setup
v3: fix comment
v4: switch to new callback method
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Sets up the GFX ring and loads ucode for GFX and Compute.

Todo:
- handle compute queue setup.

v2: add documentation
v3: integrate with latest reset changes
v4: additional init fixes
v5: scratch reg write back no longer supported on CIK
v6: properly set CP_RB0_BASE_HI
v7: rebase
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

That not only saves some power, but also solves problems with
older chips where an idle UVD block on higher clocks can
cause problems.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Just everything needed to decode videos using UVD.

v6: just all the bugfixes and support for R7xx-SI merged in one patch
v7: UVD_CGC_GATE is a write only register, lockup detection fix
v8: split out VRAM fallback changes, remove support for RV770,
    add support for HEMLOCK, add buffer sizes checks
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Let the CS module decide if we can fall back to VRAM or not.

v2: remove unintended change
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: update error message and comment
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For very large page table updates, we can exceed the
size of the ring.  To avoid this, use an IB to perform
the page table update.

v2(ck): cleanup the IB infrastructure and the use it instead
        of filling the struct ourself.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>

next_reloc function does the same thing in all ASICs with
the exception of R600 which has a special case in legacy mode.
Pull out the common function in preparation for refactoring.
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: NMarek Olšák <maraeo@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This function is not limited to r100, but it can dump a
(raw) packet for any ASIC. Rename it accordingly and move
its declaration to radeon.h
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: NMarek Olšák <maraeo@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Once we factored out radeon_cs_packet_parse function,
evergreen_cs_next_is_pkt3_nop and r600_cs_next_is_pkt3_nop
functions became identical, so they can be factored out
into a common function.
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: NMarek Olšák <maraeo@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

CS packet parse functions have a lot of in common across
all ASICs. Implement a common function and take care of
small differences between families inside the function.

This patch is a prep for major refactoring and consolidation
of CS parsing code.
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: NMarek Olšák <maraeo@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: NMarek Olšák <maraeo@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

length_dw field was assigned twice. While at it, move user_ptr
assignment together with all other assignments to p->chunks[i]
structure to make the code more readable.
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: NMarek Olšák <maraeo@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

If one (but not both) allocations of p->chunks[].kpage[]
in radeon_cs_parser_init fail, the error path will free
the successfully allocated page, but leave a stale pointer
value in the kpage[] field. This will later cause a
double-free when radeon_cs_parser_fini is called.
This patch fixes the issue by forcing both pointers to NULL
after kfree in the error path.

The circumstances under which the problem happens are very
rare. The card must be AGP and the system must run out of
kmalloc area just at the right time so that one allocation
succeeds, while the other fails.
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Cc: Herton Ronaldo Krzesinski <herton.krzesinski@canonical.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Index into chunks[] array doesn't look right.
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

In UMS mode parser->rdev is NULL, so dereferencing
will cause an oops.
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This enables the functionality added in the previous
patches.  Userspace acceleration drivers can use the
CS ioctl to submit command buffers to the async DMA
rings.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Allows us to use the DMA ring from userspace.
DMA doesn't have a good NOP packet in which to embed the
reloc idx, so userspace has to add a reloc for each
buffer used and order them to match the command stream.

v2: fix address bounds checking, reloc indexing
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Make it possible to allocate a persistent page table.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Convert #include "..." to #include <path/...> in drivers/gpu/.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
Acked-by: NDave Airlie <airlied@redhat.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: NDave Jones <davej@redhat.com>

When a VM is used on more than one ring we need to
sync to the last user.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Let's allow GCC to optimize better.

This exposed some five unused functions, but this patch doesn't remove them.
Signed-off-by: NLauri Kasanen <cand@gmx.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Make sure that the ib bo is bound and is page table is up to date
in the virtual address space.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Removing the need to wait for anything.

Still not ideal, since we need to free pt on va remove.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Move binding onto the ring, simplifying handling a bit.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Move flushing the VMs as function into the rings.
First step to make VM operations async.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Store a reference to the VM into the IB structure, that
makes calculating the IBs address a bit less complicated.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Virtual address need to be fenced to know when we can safely remove it.
This patch also properly clear the pagetable. Previously it was
serouisly broken.

Kernel 3.5/3.4 need a similar patch but adapted for difference in mutex locking.

v2: For to update pagetable when unbinding bo (don't bailout if
    bo_va->valid is true).
v3: Add kernel 3.5/3.4 comment.
v4: Fix compilation warnings.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Const IBs are executed on the CE not the CP, so we can't
fence them in the normal way.

So submit them directly before the IB instead, just as
the documentation says.

v2: keep the extra documentation
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

GPU reset need to be exclusive, one happening at a time. For this
add a rw semaphore so that any path that trigger GPU activities
have to take the semaphore as a reader thus allowing concurency.

The GPU reset path take the semaphore as a writer ensuring that
no concurrent reset take place.

v2: init rw semaphore
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Don't return success if scheduling the IB fails, otherwise
we end up with an oops in ttm_eu_fence_buffer_objects.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Try to remove or replace the cs_mutex with a
vm_mutex where it is still needed.

v2: fix locking order
v3: rebased on drm-next
Signed-off-by: NChristian König <deathsimple@vodafone.de>

Move inter ring syncing with semaphores into the
existing ring allocations, with that we need to
lock the ring mutex only once.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

radeon_cs_parser_init is called by both the legacy UMS
CS ioctl and the KMS CS ioctl.  Protect KMS specific
pieces of the code by checking that rdev is not NULL.
Reported-by: NMichael Burian <michael.burian@sbg.at>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: NDave Airlie <airlied@redhat.com>