That naming seems to make more sense, since we not
only want to run PM4 rings with it.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Replace cp, cp1 and cp2 members with just an array
of radeon_cp structs.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Give all asic and radeon_ring_* functions a
radeon_cp parameter, so they know the ring to work with.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Cayman seems to be particularly sensitive to read cache returning
old data after bind/unbind to GTT. Flush read cache for GTT range
with each fences for all new hw. Should fix several rendering glitches.
Like

V2 flush whole address space
V3 also flush shader read cache

https://bugs.freedesktop.org/show_bug.cgi?id=40221
https://bugs.freedesktop.org/show_bug.cgi?id=38022
https://bugzilla.redhat.com/show_bug.cgi?id=738790Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Move the TC flush before the texture setup to match mesa and
the ddx. Also, move the TC flush into the texture setup
function.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

blit copy functions deal with GPU pages, not CPU pages,
so rename the variables and parameters accordingly
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

reorganize the code such that only the primitives (i.e., the functions
that load the CP ring) are hardware specific; dynamically link the
primitives in a (new) pointer structure inside r600_blit at
blit initialization time so that the functions that control the blit
operations can be made common for r600 and evergreen parts
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

some 3d register bits look like magic in r600 blit functions
use predefined constants to make it more intuitive what they are
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Covert 4k pages to multiples of 64x64x4 tiles.
This is also more efficient than a scanline based
approach from the MC's perspective.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

With this patch I'm only about 50k larger with DRM debugging
enables (why is that enabled by default?!?), and slightly
smaller without.

[airlied: moved r100.c additions to radeon_ring.c]
Signed-off-by: NAndi Kleen <ak@linux.intel.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

So we used to use lpfn directly to restrict VRAM when we couldn't
access the unmappable area, however this was removed in
93225b0d as it also restricted
the gtt placements. However it was only later noticed that this
broke on some hw.

This removes the active_vram_size, and just explicitly sets it
when it changes, TTM/drm_mm will always use the real_vram_size,
and the active vram size will change the TTM size used for lpfn
setting.

We should re-work the fpfn/lpfn to per-placement at some point
I suspect, but that is too late for this kernel.

Hopefully this addresses:
https://bugs.freedesktop.org/show_bug.cgi?id=35254

v2: fix reported useful VRAM size to userspace to be correct.
Signed-off-by: NDave Airlie <airlied@redhat.com>

Unconditionally initialize the drm gem object - it's not
worth the trouble not to for the few kernel objects.

This patch only changes the place of the drm gem object,
access is still done via pointers.

v2: Uncoditionally align the size in radeon_bo_create. At
least the r600/evergreen blit code didn't to this, angering
the paranoid gem code.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Not only is linear aligned supposedly more performant,
linear general is only supported by the CB in single
slice mode.  The texture hardware doesn't support
linear general, but I think the hw automatically
upgrades it to linear aligned.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

agd5f: minor cleanups
Signed-off-by: NCédric Cano <ccano@interfaceconcept.com>
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

We were previously dropping alignment requests on the floor
when allocating buffers so we always ended up page aligned.
Certain tiling modes on 6xx+ require larger alignment which
wasn't happening before.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Cc: Jerome Glisse <j.glisse@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

We should not allocate any object into unmappable vram if we
have no means to access them which on all GPU means having the
CP running and on newer GPU having the blit utility working.

This patch limit the vram allocation to visible vram until
we have acceleration up and running.

Note that it's more than unlikely that we run into any issue
related to that as when acceleration is not woring userspace
should allocate any object in vram beside front buffer which
should fit in visible vram.

V2 use real_vram_size as mc_vram_size could be bigger than
   the actual amount of vram

[airlied: fixup r700_cp_stop case]
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Cc: stable@kernel.org
Signed-off-by: NDave Airlie <airlied@redhat.com>

This patch implements blit support for bo moves using
the 3D engine.  It uses the same method as r6xx/r7xx:
- store the base state in an IB
- emit variable state and vertex buffers to do the blit

This allows the hw to move bos using the 3D engine and allows
full use of vram beyond the pci aperture size.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Move common code to init function.

v2: make sure the bo is pinned after init as well.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Noticed while adding evergreen blit support.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This simplify and improve GPU reset for R1XX-R6XX hw, it's
not 100% reliable here are result:
- R1XX/R2XX works bunch of time in a row, sometimes it
  seems it can work indifinitly
- R3XX/R3XX the most unreliable one, sometimes you will be
  able to reset few times, sometimes not even once
- R5XX more reliable than previous hw, seems to work most
  of the times but once in a while it fails for no obvious
  reasons (same status than previous reset just no same
  happy ending)
- R6XX/R7XX are lot more reliable with this patch, still
  it seems that it can fail after a bunch (reset every
  2sec for 3hour bring down the GPU & computer)

This have been tested on various hw, for some odd reasons
i wasn't able to lockup RS480/RS690 (while they use to
love locking up).

Note that on R1XX-R5XX the cursor will disapear after
lockup haven't checked why, switch to console and back
to X will restore cursor.

Next step is to record the bogus command that leaded to
the lockup.

V2 Fix r6xx resume path to avoid reinitializing blit
module, use the gpu_lockup boolean to avoid entering
inifinite waiting loop on fence while reiniting the GPU
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Cc: Jerome Glisse <jglisse@redhat.com>
Cc: Alex Deucher <alexdeucher@gmail.com>
Signed-off-by: NMatt Turner <mattst88@gmail.com>
Reviewed-by: NCorbin Simpson <MostAwesomeDude@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

There is 3 different distinct states for an indirect buffer (IB) :
  1- free with no fence
  2- free with a fence
  3- non free (fence doesn't matter)
Previous code mixed case 2 & 3 in a single one leading to possible
catastrophique failure. This patch rework the handling and properly
separate each case. So when you get ib we set the ib as non free and
fence status doesn't matter. Fence become active (ie has a meaning
for the ib code) once the ib is scheduled or free. This patch also
get rid of the alloc bitmap as it was overkill, we know go through
IB pool list like in a ring buffer as the oldest IB is the first
one the will be free.

Fix :
https://bugs.freedesktop.org/show_bug.cgi?id=26438
and likely other bugs.

V2 remove the scheduled list, it's useless now, fix free ib scanning
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

r600_ioctl_wait_idle() now handles this.

v2: update blit fence counts
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

There is 3 different distinct states for an indirect buffer (IB) :
  1- free with no fence
  2- free with a fence
  3- non free (fence doesn't matter)
Previous code mixed case 2 & 3 in a single one leading to possible
catastrophique failure. This patch rework the handling and properly
separate each case. So when you get ib we set the ib as non free and
fence status doesn't matter. Fence become active (ie has a meaning
for the ib code) once the ib is scheduled or free. This patch also
get rid of the alloc bitmap as it was overkill, we know go through
IB pool list like in a ring buffer as the oldest IB is the first
one the will be free.

Fix :
https://bugs.freedesktop.org/show_bug.cgi?id=26438
and likely other bugs.

V2 remove the scheduled list, it's useless now, fix free ib scanning
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Only flush the gpu caches before we submit a fence.
This leads to a small performance boost when we take
the extra gpu cache flushes out of the ddx and mesa.
Once this patch is in and the drm version is bumped,
we can remove the flushes from the ddx and drm.

Also, remove the extra cache flushes from the blit
routine.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

If an error happen in r600_blit_prepare_copy report it rather
than WARNING and keeping execution. For instance if ib allocation
failed we did just warn about but then latter tried to access
NULL ib ptr causing oops. This patch also protect r600_copy_blit
with a mutex as otherwise one process might overwrite blit temporary
data with new one possibly leading to GPU lockup.

Should partialy or totaly fix:
https://bugzilla.redhat.com/show_bug.cgi?id=553279

V2 failing blit initialization is not fatal, fallback to memcpy when
this happen
V3 init blit before startup as we pin in startup, remove duplicate
code (this one was actualy tested unlike V2)
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

r600 blit cleanup path need to check if a bo was allocated before
trying to free or unpin it. This patch add this check and avoid
oops when the initialization on r6xx or r7xx hw fails.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

R300 family will hard lockup if host path read cache flush is
done through MMIO to HOST_PATH_CNTL. But scheduling same flush
through ring seems harmless. This patch remove the hdp_flush
callback and add a flush after each fence emission which means
a flush after each IB schedule. Thus we should have same behavior
without the hard lockup.

Tested on R100,R200,R300,R400,R500,R600,R700 family.

V2: Adjust fence counts in r600_blit_prepare_copy()
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

The locking & protection of radeon object was somewhat messy.
This patch completely rework it to now use ttm reserve as a
protection for the radeon object structure member. It also
shrink down the various radeon object structure by removing
field which were redondant with the ttm information. Last it
converts few simple functions to inline which should with
performances.

airlied: rebase on top of r600 and other changes.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This enables the use of interrupts on r6xx/r7xx hardware.
Interrupts are implemented via a ring buffer.  The GPU adds
interrupts vectors to the ring and the host reads them off
in the interrupt handler.  The interrupt controller requires
firmware like the CP.  This firmware must be installed and
accessble to the firmware loader for interrupts to function.

MSIs don't seem to work on my RS780.  They work fine on all
my discrete cards.  I'm not sure about other RS780s or
RS880s.  I've disabled MSIs on RS780 and RS880, but it would
probably be worth checking on some other systems.

v2 - fix some checkpatch.pl problems;
     re-read the disp int status reg if we restart the ih;

v3 - remove the irq handler if r600_irq_init() fails;
     remove spinlock in r600_ih_ring_fini();
     move ih rb overflow check to r600_get_ih_wptr();
     move irq ack to separate function;
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Lots of cases were wrong or missing.

v2: rebased against drm-next
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This reverts commit 49c458e5. It
seems to have some side effects in the non-kms cases.
Signed-off-by: NDave Airlie <airlied@redhat.com>

There is no need to assign vb before you know that space is available.

[agd5f: adapted for kernel tree.]
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

[agd5f: also fix the non-kms path]
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>

a) don't zero gart table on gart enable
b) move pinning shader object into resume path
c) unpin shader object on suspend
d) set cp ready to false after cp shutdown on suspend.
Signed-off-by: NDave Airlie <airlied@redhat.com>

Some people were seeing
*ERROR* radeon: writting more dword to ring than expected
after certain blits, the loops calculation didn't take
into account that we do a separate blit for the remainder
after doing the aligned blits.
Acked-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

We sometimes lock IB then the ring and sometimes the ring then
the IB. This is mostly due to the IB locking not being well defined
about what data in the structs it actually locks. Define what I
believe is the correct behaviour and gets rid of the lock dep ordering
warning.
Signed-off-by: NDave Airlie <airlied@redhat.com>

rv6xx emits two extra dwords in the render target setup.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@linux.ie>

select the correct max number of bytes per blit based
on whether the size is multiple of 4 bytes.  This
determines whether we can use 8 or 32 bit pixels for
the blit.

airlied: also merged the IB padding patch +
correcting the VS offset for context
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>