Set asic specific gart pte flags in the gmc IP module for
each asic.
Signed-off-by: NAlex Xie <AlexBin.Xie@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Necessary for new asics.
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When the fast blit path fails while attempting to move a buffer from RAM
to VRAM, we fall back to a CPU-based memcpy that cannot handle split VRAM
buffers. Instead of crashing, simply fail the buffer move.

Ideally, we would teach TTM about split buffers so that the fallback still
works in this case, but that is quite involved. So for now, apply the
simplest possible fix.

Fixes: 40361bb1704b ("drm/amdgpu: add VRAM manager v2")
Signed-off-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Here, If ioremap_nocache will fail. It will return NULL.
Kernel can run into a NULL-pointer dereference.
This error check will avoid NULL pointer dereference.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NArvind Yadav <arvind.yadav.cs@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The additional housekeeping had too much CPU overhead,
let's use the BO priorities instead.

agd: also revert hibmc changes
Reviewed-by: NSinclair Yeh <syeh@vmware.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-and-Tested-by: NRoger.He <Hongbo.He@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Keeping groups of BOs on the LRU is to time consuming on command submission.

Instead use the newly added BO priority to give a certain eviction order.

v2: agd: trivial warning fix
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NRoger.He <Hongbo.He@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This way the driver can specify a priority for a BO which has the effect that
a BO is only evicted when all other BOs with a lower priority are evicted
first.
Reviewed-by: NSinclair Yeh <syeh@vmware.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NRoger.He <Hongbo.He@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This catches evictions of shadow page tables from the GART. Since shadow
page tables are always stored in system memory, amdgpu_bo_move is never
called for them.

This fixes a crash during command submission that occurs when only a shadow
page table and no other BOs were evicted since the last submission.

Fixes: 1baa439f ("drm/amdgpu: allocate shadow for pd/pt bo V2")
Signed-off-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Including all drivers. I thought about keeping small compat functions
to avoid having to change all drivers. But I really like the
drm_printer idea, so figured spreading it more widely is a good thing.

v2: Review from Chris:
- Natural argument order and better name for drm_mm_print.
- show_mm() macro in the selftest.

Cc: Rob Clark <robdclark@gmail.com>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Tomi Valkeinen <tomi.valkeinen@ti.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Jyri Sarha <jsarha@ti.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1483009764-8281-1-git-send-email-daniel.vetter@ffwll.ch

This reverts commit a693e050.

This breaks on systems with huge amounts of system memory as we do not have
enough vram to store the page tables.  Additionally, this is less of an issue
with the recent gtt manager changes.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

The mem start is relative to the domain in the address space, so this
worked only when VRAM was mapped at offset 0.

It also didn't handled multiple drm_mm_nodes for split VRAM.

v2: rebase on dma_fence renaming
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This way we can correctly check split VRAM buffers as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This way the driver can decide if it is valuable to evict a BO or not.

The current implementation is added as default to all existing drivers.

v2: fix some typos found during internal testing
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It's completely pointless to have two pointers to the
device in the same structure.

v2: rename function to amdgpu_ttm_adev, fix typos
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Split VRAM allocations into 4MB blocks.

v2: fix typo in comment, some suggested cleanups
v3: document how to disable the feature, fix rebase issue
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NEdward O'Callaghan <funfunctor@folklore1984.net>
Tested-by: NMike Lothian <mike@fireburn.co.uk>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This allows us to move scattered buffers around.

v2: fix a couple of typos, handle scattered to scattered moves as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NEdward O'Callaghan <funfunctor@folklore1984.net>
Tested-by: NMike Lothian <mike@fireburn.co.uk>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add a flag noting that a BO must be created using linear VRAM
and set this flag on all in kernel users where appropriate.

Hopefully I haven't missed anything.

v2: add it in a few more places, fix CPU mapping.
v3: rename to VRAM_CONTIGUOUS, fix typo in CS code.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NEdward O'Callaghan <funfunctor@folklore1984.net>
Tested-by: NMike Lothian <mike@fireburn.co.uk>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

I plan to usurp the short name of struct fence for a core kernel struct,
and so I need to rename the specialised fence/timeline for DMA
operations to make room.

A consensus was reached in
https://lists.freedesktop.org/archives/dri-devel/2016-July/113083.html
that making clear this fence applies to DMA operations was a good thing.
Since then the patch has grown a bit as usage increases, so hopefully it
remains a good thing!

(v2...: rebase, rerun spatch)
v3: Compile on msm, spotted a manual fixup that I broke.
v4: Try again for msm, sorry Daniel

coccinelle script:
@@

@@
- struct fence
+ struct dma_fence
@@

@@
- struct fence_ops
+ struct dma_fence_ops
@@

@@
- struct fence_cb
+ struct dma_fence_cb
@@

@@
- struct fence_array
+ struct dma_fence_array
@@

@@
- enum fence_flag_bits
+ enum dma_fence_flag_bits
@@

@@
(
- fence_init
+ dma_fence_init
|
- fence_release
+ dma_fence_release
|
- fence_free
+ dma_fence_free
|
- fence_get
+ dma_fence_get
|
- fence_get_rcu
+ dma_fence_get_rcu
|
- fence_put
+ dma_fence_put
|
- fence_signal
+ dma_fence_signal
|
- fence_signal_locked
+ dma_fence_signal_locked
|
- fence_default_wait
+ dma_fence_default_wait
|
- fence_add_callback
+ dma_fence_add_callback
|
- fence_remove_callback
+ dma_fence_remove_callback
|
- fence_enable_sw_signaling
+ dma_fence_enable_sw_signaling
|
- fence_is_signaled_locked
+ dma_fence_is_signaled_locked
|
- fence_is_signaled
+ dma_fence_is_signaled
|
- fence_is_later
+ dma_fence_is_later
|
- fence_later
+ dma_fence_later
|
- fence_wait_timeout
+ dma_fence_wait_timeout
|
- fence_wait_any_timeout
+ dma_fence_wait_any_timeout
|
- fence_wait
+ dma_fence_wait
|
- fence_context_alloc
+ dma_fence_context_alloc
|
- fence_array_create
+ dma_fence_array_create
|
- to_fence_array
+ to_dma_fence_array
|
- fence_is_array
+ dma_fence_is_array
|
- trace_fence_emit
+ trace_dma_fence_emit
|
- FENCE_TRACE
+ DMA_FENCE_TRACE
|
- FENCE_WARN
+ DMA_FENCE_WARN
|
- FENCE_ERR
+ DMA_FENCE_ERR
)
 (
 ...
 )
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NGustavo Padovan <gustavo.padovan@collabora.co.uk>
Acked-by: NSumit Semwal <sumit.semwal@linaro.org>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20161025120045.28839-1-chris@chris-wilson.co.uk

This removes the 'write' and 'force' from get_user_pages() and replaces
them with 'gup_flags' to make the use of FOLL_FORCE explicit in callers
as use of this flag can result in surprising behaviour (and hence bugs)
within the mm subsystem.
Signed-off-by: NLorenzo Stoakes <lstoakes@gmail.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Acked-by: NJesper Nilsson <jesper.nilsson@axis.com>
Acked-by: NMichal Hocko <mhocko@suse.com>
Reviewed-by: NJan Kara <jack@suse.cz>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This fixes a memory leak since binding GTT only on demand.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It's pretty pointless to get the offset first and then initialize it.

Should fix issues with the new GTT manager.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This bug seems to be present for a very long time.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Just to cleanup some radeon leftovers.

sed -i "s/rbo/abo/g" drivers/gpu/drm/amd/amdgpu/*.c
sed -i "s/rbo/abo/g" drivers/gpu/drm/amd/amdgpu/*.h

v2: rebased
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Only allocate address space when we really need it.

v2: fix a typo, add correct function description,
    stop leaking the node in the error case.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Rather than using "struct file*", use "struct drm_file*" as tag VM tag for
BOs. This will pave the way for "struct drm_file*" without any "struct
file*" back-pointer.
Signed-off-by: NDavid Herrmann <dh.herrmann@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20160901124837.680-3-dh.herrmann@gmail.com

Use an explicit limit when moving things around to
force address space allocation.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This way the address space gets allocated immediately.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Doesn't make much sense to clear the GTT when it was never filled.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Use error label to decrease global item unref and make codes more
readable.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We don't really need the GTT table any more most of the time. So bind it
only on demand.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Acked-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Otherwise we run into problems on 32bit systems with more than 4GB GART.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

V2:
a. gart recovery should be ahead of ring test.
b. rename to amdgpu_ttm_recover_gart
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

V2:
spin lock instead of mutex for gtt list
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Adding a BO can make it the insertion point for larger sizes as well.

v2: add a comment about the guard structure.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NFelix Kuehling <felix.kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Adding a BO can make it the insertion point for larger sizes as well.

v2: add a comment about the guard structure.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NFelix Kuehling <felix.kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NFlora Cui <Flora.Cui@amd.com>
Reviewed-by: NKen Wang <Qingqing.Wang@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This bug seems to be present for a very long time.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org