This patch allows the CPU to map the stolen vram segment
directly rather than going through the PCI BAR.  This
significantly improves performance for certain workloads with
a properly patched ddx.

Use radeon.fastfb=1 to enable it (disabled by default).
Currently only supported on RS690, but support for RS780/880
and newer APUs may be added eventually.
Signed-off-by: NSamuel Li <samuel.li@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add a per-asic MC (memory controller) mask which holds the
mak address mask the asic is capable of.  Use this when
calculating the vram and gtt locations rather using asic
specific functions or limiting everything to 32 bits.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Cleans up the code for future asics

v2: rebase, fix some missing radeon_asic updates
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is required to get the reference clock used
by the gfx engine for things like timestamps. Fixes
support for GL extensions the use timestamps on
certain boards.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Simplify the Radeon prime implementation by using the default behavior provided
by drm_gem_prime_import and drm_gem_prime_export.

v2:
- Rename functions to radeon_gem_prime_get_sg_table and
  radeon_gem_prime_import_sg_table.
- Delete the now-unused vmapping_count variable.
Signed-off-by: NAaron Plattner <aplattner@nvidia.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: David Airlie <airlied@linux.ie>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Oland is a new asic in the SI family.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For very large page table updates, we can exceed the
size of the ring.  To avoid this, use an IB to perform
the page table update.

v2(ck): cleanup the IB infrastructure and the use it instead
        of filling the struct ourself.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>

Used by all asic families from r600+.
Flag for the vbios and later instances of the driver
that the GPU is hung.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This adds further flags for fine grained reset.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

next_reloc function does the same thing in all ASICs with
the exception of R600 which has a special case in legacy mode.
Pull out the common function in preparation for refactoring.
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: NMarek Olšák <maraeo@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This function is not limited to r100, but it can dump a
(raw) packet for any ASIC. Rename it accordingly and move
its declaration to radeon.h
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: NMarek Olšák <maraeo@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

vline packet parsing function for R600 and Evergreen+ are
the same, except that they use different registers. Factor
out the algorithm into a common function that uses register
table passed from ASIC-specific caller.

This reduces ASIC-specific function to (trivial) setup
of register table and call into the common function.
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: NMarek Olšák <maraeo@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Once we factored out radeon_cs_packet_parse function,
evergreen_cs_next_is_pkt3_nop and r600_cs_next_is_pkt3_nop
functions became identical, so they can be factored out
into a common function.
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: NMarek Olšák <maraeo@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We now have a common radeon_cs_packet_parse function
that is good for all ASICs. Hook it up and eliminate
ASIC-specific versions.
Signed-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Reviewed-by: NMarek Olšák <maraeo@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This reverts commit d025e9e2.

This causes corruption for a number of users and needs further
investigation in the next cycle.
https://bugzilla.kernel.org/show_bug.cgi?id=52491
https://bugs.freedesktop.org/show_bug.cgi?id=58659
http://lists.freedesktop.org/archives/dri-devel/2013-January/032961.htmlSigned-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The idea here is to move to a finer grained reset.
In some cases we may not need reset every block, and
in other cases we may not need to re-init the entire
asic.
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

radeon_fence_wait_empty_locked should not trigger GPU reset as no
place where it's call from would benefit from such thing and it
actually lead to a kernel deadlock in case the reset is triggered
from pm codepath. Instead force ring completion in place where it
makes sense or return early in others.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Force all fence to signal if GPU reset failed so no process get stuck
on waiting fence.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Allows us to use the DMA ring from userspace.
DMA doesn't have a good NOP packet in which to embed the
reloc idx, so userspace has to add a reloc for each
buffer used and order them to match the command stream.

v2: fix address bounds checking, reloc indexing
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

With the new per-crtc locking mutliple set-cursor calls could happen
in parallel. Out of sheer paranoia I've opted for an irqsave spinlock.
But if there's indeed an access from interrupt contexts to these regs
it's already broken with the old code, so this can likely just be
reduced to a normal spinlock. Otoh the pageflip completion happens
from the vblank irq handler ...
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Just refactoring to make the next patche simpler. Now all indirect register
access in the new modesetting driver should go through the r100_mm_(w|r)reg
fucntions.

RADEON_READ_MM from the old driver seems to be totally unused, so just kill
it.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The bo creation placement is where the bo will be. Instead of trying
to move bo at each command stream let this work to another worker
thread that will use more advance heuristic.

agd5f: remove leftover unused variable
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Pretty much the same as cayman.  Some changes to the copy
packets.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

There are 2 async DMA engines on cayman, one at 0xd000 and
one at 0xd800.  The programming interface is the same as
evergreen however there are some changes to the commands
for using vmids.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Uses the new multi-ring infrastucture.  6xx/7xx has a single
async DMA ring.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Make it possible to allocate a persistent page table.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We want to use VMs without the IB pool in the future.

v2: also remove it from radeon_vm_finish.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Based on Dmitries work, but splitting the code into page
directory and page table handling makes it far more
readable and (hopefully) more reliable.

Allocations of page tables are made from the SA on demand,
that should still work fine since all page tables are of
the same size.

Also using the fact that allocations from the SA are mostly
continuously (except for end of buffer wraps and under very
high memory pressure) to group updates send to the chipset
specific code into larger chunks.

v3: mostly a rewrite of Dmitries previous patch.
v4: fix some typos and coding style
Signed-off-by: NDmitry Cherkasov <Dmitrii.Cherkasov@amd.com>
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Tested-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Pass the vm and ring index rather than an IB.  This allows
us to use the vm_flush interface for non-IB cases in the
future.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

PDE/PTE update code uses CP ring for memory writes.
All page table entries are preallocated for now in alloc_pt().

It is made as whole because it's hard to divide it to several patches
that compile and doesn't break anything being applied separately.

Tested on cayman card.

v2: rebased on top of "refactor set_page chipset interface v3",
    code cleanups

v3: switched offsets calc macros to inline funcs where possible,
    remove pd_addr from radeon_vm, switched RADEON_BLOCK_SIZE define,
    to 9 (and PTE_COUNT to 1 << BLOCK_SIZE)

v4 (ck): move "incr" documentation to previous patch, cleanup and
         document RADEON_VM_* constants, change commit message to
         our usual format, simplify patch allot by removing
         everything current not necessary, disable SI workaround.

v5: (agd5f): Fix typo in tables_size calculation in
             radeon_vm_alloc_pt().  Second line should have been
             '+=' rather than '='.

v6: fix npdes calculation. In scenario when pfns to be mapped overlap
two PDE spans:

   +-----------+-------------+
   | PDE span  | PDE span    |
   +-----------+----+--------+
          |         |
          +---------+
          | pfns    |
          +---------+

the following npdes calculation gives incorrect result:

npdes = (nptes >> RADEON_VM_BLOCK_SIZE) + 1;

For the case above picture it should give npdes = 2, but gives one.

This patch corrects it by rounding last pfn up to 512 border,
first - down to 512 border and then subtracting and dividing by 512.

v7: Make npde calculation clearer, fix ndw calculation.

v8: (agd5f): reserve enough for 2 full VM PTs, add some
             additional comments.

v9: fix typo in npde calculation
Signed-off-by: NDmitry Cherkasov <Dmitrii.Cherkasov@amd.com>
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Cleanup the interface in preparation for hierarchical page tables.

v2: add incr parameter to set_page for simple scattered PTs uptates
    added PDE-specific flags to r600_flags and radeon_drm.h
    removed superfluous value masking with 0xffffffff

v3: removed superfluous bo_va->valid checking
    changed R600_PTE_VALID to R600_ENTRY_VALID to handle PDE too

v4 (ck): fix indention style, rework and fix typos in commit message,
         add documentation for incr parameter, also use incr
         parameter for system pages

v5 (agd5f): use upper_32_bits() and minor white space fixes
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NDmitry Cherkassov <Dmitrii.Cherkasov@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Read back the backlight level from the hw.
Needed for proper backlight restoration on resume.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Roughly based on how nouveau is handling it. Instead of
adding the bo_va when the address is set add the bo_va
when the handle is opened, but set the address to zero
until userspace tells us where to place it.

This fixes another bunch of problems with glamor.

v2: agd5f: fix build after dropping patch 7/8.
Signed-off-by: NChristian König <deathsimple@vodafone.de>

It doesn't really belong into the object functions,
also rename it to avoid collisions with struct radeon_bo_va.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Even GPUs can have a null pointer dereference, so move
the IB pool to another offset to catch those.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Currently doing the update with the CP.

v2: Rebased on Jeromes bugfix. Make validity comparison
    more human readable.
Signed-off-by: NChristian König <deathsimple@vodafone.de>

Makes it easier to move it into the rings.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Removing the need to wait for anything.

Still not ideal, since we need to free pt on va remove.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Move binding onto the ring, simplifying handling a bit.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>