Adds an extra argument to radeon_bo_create, which is only used in radeon_prime.c.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Really, the legacy buffer api should be dead, especially for all these
newfangled drivers. I suspect this is copypasta from the transitioning
days, which probably originated in radeon.

Cc: "Christian König" <christian.koenig@amd.com>
Cc: David Herrmann <dh.herrmann@gmail.com>
Cc: Rashika <rashika.kheria@gmail.com>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Fabian Frederick <fabf@skynet.be>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Alexandre Courbot <acourbot@nvidia.com>
Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Cc: Christian Engelmayer <cengelma@gmx.at>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Reviewed-by: NDavid Herrmann <dh.herrmann@gmail.com>
Acked-by: NBen Skeggs <bskeggs@redhat.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

AGP mappings are not cache coherent, so userptr support
won't work. Additional to that the AGP implementation uses
a different ttm_tt container structure so we run into
problems if we cast the pointer without checking if it's
the right type.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This allows us to specify if we want to sync to
the shared fences of a reservation object or not.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>

This allows us to more fine grained specify where to place the buffer object.

v2: rebased on drm-next, add bochs changes as well
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Avoid problems with writeback by limiting userptr to anonymous memory.

v2: add commit and code comments
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This patch adds an IOCTL for turning a pointer supplied by
userspace into a buffer object.

It imposes several restrictions upon the memory being mapped:

1. It must be page aligned (both start/end addresses, i.e ptr and size).

2. It must be normal system memory, not a pointer into another map of IO
space (e.g. it must not be a GTT mmapping of another object).

3. The BO is mapped into GTT, so the maximum amount of memory mapped at
all times is still the GTT limit.

4. The BO is only mapped readonly for now, so no write support.

5. List of backing pages is only acquired once, so they represent a
snapshot of the first use.

Exporting and sharing as well as mapping of buffer objects created by
this function is forbidden and results in an -EPERM.

v2: squash all previous changes into first public version
v3: fix tabs, map readonly, don't use MM callback any more
v4: set TTM_PAGE_FLAG_SG so that TTM never messes with the pages,
    pin/unpin pages on bind/unbind instead of populate/unpopulate
v5: rebased on 3.17-wip, IOCTL renamed to userptr, reject any unknown
    flags, better handle READONLY flag, improve permission check
v6: fix ptr cast warning, use set_page_dirty/mark_page_accessed on unpin
v7: add warning about it's availability in the API definition
v8: drop access_ok check, fix VM mapping bits
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v4)
Reviewed-by: Jérôme Glisse <jglisse@redhat.com> (v4)
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: fix rebase onto drm-fixes
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

With dev->anon_inode we have a global address_space ready for operation
right from the beginning. Therefore, there is no need to do a delayed
setup with TTM. Instead, set dev_mapping during initialization in
ttm_bo_device_init() and remove any "if (dev_mapping)" conditions.

Cc: Dave Airlie <airlied@redhat.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Cc: Alex Deucher <alexdeucher@gmail.com>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDavid Herrmann <dh.herrmann@gmail.com>

DRM drivers share a common address_space across all character-devices of a
single DRM device. This allows simple buffer eviction and mapping-control.
However, DRM core currently waits for the first ->open() on any char-dev
to mark the underlying inode as backing inode of the device. This delayed
initialization causes ugly conditions all over the place:
  if (dev->dev_mapping)
    do_sth();

To avoid delayed initialization and to stop reusing the inode of the
char-dev, we allocate an anonymous inode for each DRM device and reset
filp->f_mapping to it on ->open().
Signed-off-by: NDavid Herrmann <dh.herrmann@gmail.com>

Building radeon_ttm.o on 32 bit x86 triggers a warning:
    In file included from include/asm-generic/bug.h:13:0,
                     from [...]/arch/x86/include/asm/bug.h:38,
                     from include/linux/bug.h:4,
                     from include/drm/drm_mm.h:39,
                     from include/drm/drm_vma_manager.h:26,
                     from include/drm/ttm/ttm_bo_api.h:35,
                     from drivers/gpu/drm/radeon/radeon_ttm.c:32:
    drivers/gpu/drm/radeon/radeon_ttm.c: In function 'radeon_ttm_gtt_read':
    include/linux/kernel.h:712:17: warning: comparison of distinct pointer types lacks a cast [enabled by default]
      (void) (&_min1 == &_min2);  \
                     ^
    drivers/gpu/drm/radeon/radeon_ttm.c:938:22: note: in expansion of macro 'min'
       ssize_t cur_size = min(size, PAGE_SIZE - off);
                          ^

Silence this warning by using min_t(). Since cur_size will never be
negative and its upper bound is PAGE_SIZE, we can change its type to
size_t and use min_t(size_t, [...]) here.
Signed-off-by: NPaul Bolle <pebolle@tiscali.nl>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Without this, a bo may get created in the cpu-inaccessible vram.
Before the CP engines get setup, all copies are done via cpu memcpy.

This means that the cpu tries to read from inaccessible memory, fails,
and the radeon module proceeds to disable acceleration.

Doing this has no downsides, as the real VRAM size gets set as soon as the
CP engines get init.

This is a candidate for 3.14 fixes.

v2: Add comment on why the function is used
Signed-off-by: NLauri Kasanen <cand@gmx.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org

The statistics are:
- VRAM usage in bytes
- GTT usage in bytes
- number of bytes moved by TTM

The last one is actually a counter, so you need to sample it before and after
command submission and take the difference.

This is useful for finding performance bottlenecks. Userspace queries are
also added.

v2: use atomic64_t
Signed-off-by: NMarek Olšák <marek.olsak@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

v2: add default_llseek
v3: set inode size in the open callback
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Not very fast, but makes it possible to access even the
normally inaccessible parts of VRAM from userspace.

v2: use MM_INDEX_HI for >2GB mem access, add default_llseek
v3: set inode size in the open callback
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Otherwise we not necessary export the right information.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Most place actually want to just check for dev->agp (most do, but a
few don't so this fixes a few potential NULL derefs). The only
exception is the agp init code which should check for the AGP driver
feature flag.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

GEM does already a good job in tracking access to gem buffers via handles
and drm_vma access management. However, TTM drivers currently do not
verify this during mmap().

TTM provides the verify_access() callback to test this. So fix all drivers
to actually call into gem+vma to verify access instead of always returning
0.

All drivers assume that user-space can only get access to TTM buffers via
GEM handles. So whenever the verify_access() callback is called from
ttm_bo_mmap(), the buffer must have a valid embedded gem object. This is
true for all TTM+GEM drivers. But that's why this patch doesn't touch pure
TTM drivers (ie, vmwgfx).

v2: Switch to drm_vma_node_verify_access() to correctly return -EACCES if
    access was denied.

Cc: Dave Airlie <airlied@redhat.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Maarten Lankhorst <maarten.lankhorst@canonical.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: NDavid Herrmann <dh.herrmann@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Add ULL prefix to avoid overflow.
Signed-off-by: NNiels Ole Salscheider <niels_ole@salscheider-online.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

On tile architecture (with "make allyesconfig") including
<linux/swiotlb.h> is required to call swiotlb_nr_tbl().
Signed-off-by: NChris Metcalf <cmetcalf@tilera.com>
Acked-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

On tile architecture (with "make allyesconfig") including
<linux/swiotlb.h> is required to call swiotlb_nr_tbl().
Signed-off-by: NChris Metcalf <cmetcalf@tilera.com>
Acked-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>

All items on the lru list are always reservable, so this is a stupid
thing to keep. Not only that, it is used in a way which would
guarantee deadlocks if it were ever to be set to block on reserve.

This is a lot of churn, but mostly because of the removal of the
argument which can be nested arbitrarily deeply in many places.

No change of code in this patch except removal of the no_wait_reserve
argument, the previous patch removed the use of no_wait_reserve.

v2:
 - Warn if -EBUSY is returned on reservation, all objects on the list
   should be reservable. Adjusted patch slightly due to conflicts.
v3:
 - Focus on no_wait_reserve removal only.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-By: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-By: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Let's allow GCC to optimize better.

This exposed some five unused functions, but this patch doesn't remove them.
Signed-off-by: NLauri Kasanen <cand@gmx.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Setting dev_mapping (pointer to the address_space structure
used for memory mappings) to the address_space of the first
opener's inode and then failing if other openers come in
through a different inode has a few restrictions that are
eliminated by this patch.

If we already have valid dev_mapping and we spot an opener
with different i_node, we force its i_mapping pointer to the
already established address_space structure (first opener's
inode). This will make all mappings from drm device hang off
the same address_space object.

Some benefits (things that now work and didn't work
before) of this patch are:

 * user space can mknod and use any number of device
   nodes and they will all work fine as long as the major
   device number is that of the drm module.
 * user space can even remove the first opener's device
   nodes and mknod the new one and the applications and
   windowing system will still work.
 * GPU drivers can safely assume that dev->dev_mapping is
   correct address_space and just blindly copy it
   into their (private) bdev.dev_mapping

For reference, some discussion that lead to this patch can
be found here:

http://lists.freedesktop.org/archives/dri-devel/2012-April/022283.htmlSigned-off-by: NIlija Hadzic <ihadzic@research.bell-labs.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

It is a rw_semaphore now and only write locked
while changing the clock. Also the lock is renamed
to better reflect what it is protecting.

v2: Keep the ttm_vm_ops on IGPs
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

Move inter ring syncing with semaphores into the
existing ring allocations, with that we need to
lock the ring mutex only once.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

It is completely unnecessary to create fences
before they are emitted, so remove it and a bunch
of checks if fences are emitted or not.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

This adds prime->fd and fd->prime support to radeon.
It passes the sg object to ttm and then populates
the gart entries using it.

Compile tested only.

v2: stub kmap + use new helpers + add reimporting
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

We can now protected the semaphore ram by a
fence, so free it immediately.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Rings need to lock in order, otherwise
the ring subsystem can deadlock.

v2: fix error handling and number of locked doublewords.
v3: stop creating unneeded semaphores.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

tidy up the radeon_asic struct, handle multiple
rings better.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König<christian.koenig@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

ttm tt rework modified the way we allocate and populate the
ttm_tt structure, the AGP side was missing some bit to properly
work. Fix those and fix radeon and nouveau AGP support.

Tested on radeon only so far.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

We need to synchronize across rings when doing a bo move to make
sure we the buffer is idle if it's in use by a different ring than
the ring doing the move.

v2: fix fence setup for bo moves

v3: add missing ring lock/unlock
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

That naming seems to make more sense, since we not
only want to run PM4 rings with it.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Replace cp, cp1 and cp2 members with just an array
of radeon_cp structs.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

For supporting multiple CP ring buffers, async DMA
engines and UVD.  We still need a way to synchronize
between engines.

v2 initialize unused fence driver ring to avoid issue in
   suspend/unload
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>