When the mutex is locked just in the moment we copy it we end up with a
warning that we release a locked mutex.

Fix this by properly reinitializing the mutex.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
CC: stable@vger.kernel.org
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We don't use compound pages at the moment. Take this into account when
freeing them.
Signed-off-by: NChristian König <christian.koenig@amd.comd>
Reviewed-and-Tested-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add a new huge page pool and try to allocate from it when it makes sense.

v2: avoid compound pages for now
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Make it easier to add huge page pool.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Try to allocate huge pages when it makes sense.

v2: avoid compound pages for now
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We need to figure out first how to correctly map them into the CPU page tables.

bug: https://bugs.freedesktop.org/show_bug.cgi?id=103138Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of mapping them bit by bit map/unmap all consecutive
pages as in one call.

v2: test for consecutive pages instead of using compound page order.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Totally surprisingly this is more efficient than doing it page by page.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Try to allocate huge pages when it makes sense.

v2: fix comment and use ifdef
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Correctly handle different page sizes in the memory accounting.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Nobody is actually using that, remove it.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Remove unused defines and variables. Also stop computing the
gfp_flags when they aren't used.

No intended functional change.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We need to free the reservation object before we take the BO
from the delayed delete list.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMonk Liu <monk.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

With shared reservation objects it is possible that we want to temporary
kmap an BO while it is still on the swap list.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NLeo Liu <leo.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

With shared reservation objects __ttm_bo_reserve() can easily fail even on
destroyed BOs. This prevents correct handling when we need to individualize
the reservation object.

Fix this by individualizing the object before even trying to reserve it.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Fixed kbuild errors when IOMMU/SWIOTLB are disabled.
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Reviewed-by: NThierry Reding <treding@nvidia.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

(v2): Drop Makefile change too.

These functions replace a section of common code found
in radeon/amdgpu drivers (and possibly others) as part
of the ttm_tt_*populate() callbacks.

v2: squash in fix for sw iommu from Tom
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Also exports two functions that vendor drivers can call
to trace DMA mappings.  This is meant to help translate
IOMMU mappings of bus addresses back to physical pages.

Used by the umr amdgpu debugger for instance.
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

(v2): Use dev_name() to get PCI path instead.
(v3): Use correct types for dma/phys addresses

we observe abnormal number from:
/sys/devices/virtual/drm/amdttm/buffer_objects/bo_count

bo_count is atomic_inc which is "int" type,
shouldn't explicitly turn it to unsigned long.
Signed-off-by: NMonk Liu <monk.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NMonk Liu <monk.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

we observe abnormal number from:
/sys/devices/virtual/drm/amdttm/buffer_objects/bo_count

bo_count is atomic_inc which is "int" type,
shouldn't explicitly turn it to unsigned long.
Signed-off-by: NMonk Liu <monk.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NMonk Liu <monk.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Fixes a false positive from might_sleep(). The reservation object is freshly
initialized, so nobody else can hold the mutex but the function is
called from atomic context.

v2: Correctly invert the check as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When fail to get needed page for pool, need to put allocated pages
into pool. But current code has a miscalculation of allocated pages,
correct it.
Signed-off-by: NXiangliang.Yu <Xiangliang.Yu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMonk Liu <monk.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Provide the drm printer directly instead of just the callback.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Use the BOs reservation object when it is put on the ddelete list. This way we
avoid delaying freeing up the BO because of new fences on the reservation object.

This is used by dma-buf and amdgpu's VM page tables.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

With shared reservation objects the assumption that no fence
could have been added isn't true any more.

Additional to that the BO is about to be destroyed, so removing the
fences now has no advantage whatsoever.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Allows gdb to access contents of user mode mapped BOs. System memory
is handled by TTM using kmap. Other memory pools require a new driver
callback in ttm_bo_driver.

v2:
* kmap only one page at a time
* swap in BO if needed
* make driver callback more generic to handle private memory pools
* document callback return value
* WARN_ON -> WARN_ON_ONCE
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Since video memory needs to be accessed decrypted, be sure that the
memory encryption mask is not set for the video ranges.
Signed-off-by: NTom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Reviewed-by: NBorislav Petkov <bp@suse.de>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Larry Woodman <lwoodman@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Matt Fleming <matt@codeblueprint.co.uk>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Toshimitsu Kani <toshi.kani@hpe.com>
Cc: kasan-dev@googlegroups.com
Cc: kvm@vger.kernel.org
Cc: linux-arch@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: linux-efi@vger.kernel.org
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/a19436f30424402e01f63a09b32ab103272acced.1500319216.git.thomas.lendacky@amd.comSigned-off-by: NIngo Molnar <mingo@kernel.org>

We unref the man->move fence in ttm_bo_clean_mm() and then call
ttm_bo_force_list_clean() which waits on it, except the refcount is now
zero so a warning is generated (or worse):

[149492.279301] refcount_t: increment on 0; use-after-free.
[149492.279309] ------------[ cut here ]------------
[149492.279315] WARNING: CPU: 3 PID: 18726 at lib/refcount.c:150 refcount_inc+0x2b/0x30
[149492.279315] Modules linked in: vhost_net vhost tun x86_pkg_temp_thermal crc32_pclmul ghash_clmulni_intel efivarfs amdgpu(
-) i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm
[149492.279326] CPU: 3 PID: 18726 Comm: rmmod Not tainted 4.12.0-rc5-drm-next-4.13-ttmpatch+ #1
[149492.279326] Hardware name: Gigabyte Technology Co., Ltd. Z97X-UD3H-BK/Z97X-UD3H-BK-CF, BIOS F6 06/17/2014
[149492.279327] task: ffff8804ddfedcc0 task.stack: ffffc90008d20000
[149492.279329] RIP: 0010:refcount_inc+0x2b/0x30
[149492.279330] RSP: 0018:ffffc90008d23c30 EFLAGS: 00010286
[149492.279331] RAX: 000000000000002b RBX: 0000000000000170 RCX: 0000000000000000
[149492.279331] RDX: 0000000000000000 RSI: ffff88051ecccbe8 RDI: ffff88051ecccbe8
[149492.279332] RBP: ffffc90008d23c30 R08: 0000000000000001 R09: 00000000000003ee
[149492.279333] R10: ffffc90008d23bb0 R11: 00000000000003ee R12: ffff88043aaac960
[149492.279333] R13: ffff8805005e28a8 R14: 0000000000000002 R15: ffff88050115e178
[149492.279334] FS:  00007fc540168700(0000) GS:ffff88051ecc0000(0000) knlGS:0000000000000000
[149492.279335] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[149492.279336] CR2: 00007fc3e8654140 CR3: 000000027ba77000 CR4: 00000000001426e0
[149492.279337] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[149492.279337] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[149492.279338] Call Trace:
[149492.279345]  ttm_bo_force_list_clean+0xb9/0x110 [ttm]
[149492.279348]  ttm_bo_clean_mm+0x7a/0xe0 [ttm]
[149492.279375]  amdgpu_ttm_fini+0xc9/0x1f0 [amdgpu]
[149492.279392]  amdgpu_bo_fini+0x12/0x40 [amdgpu]
[149492.279415]  gmc_v7_0_sw_fini+0x32/0x40 [amdgpu]
[149492.279430]  amdgpu_fini+0x2c9/0x490 [amdgpu]
[149492.279445]  amdgpu_device_fini+0x58/0x1b0 [amdgpu]
[149492.279461]  amdgpu_driver_unload_kms+0x4f/0xa0 [amdgpu]
[149492.279470]  drm_dev_unregister+0x3c/0xe0 [drm]
[149492.279485]  amdgpu_pci_remove+0x19/0x30 [amdgpu]
[149492.279487]  pci_device_remove+0x39/0xc0
[149492.279490]  device_release_driver_internal+0x155/0x210
[149492.279491]  driver_detach+0x38/0x70
[149492.279493]  bus_remove_driver+0x4c/0xa0
[149492.279494]  driver_unregister+0x2c/0x40
[149492.279496]  pci_unregister_driver+0x21/0x90
[149492.279520]  amdgpu_exit+0x15/0x406 [amdgpu]
[149492.279523]  SyS_delete_module+0x1a8/0x270
[149492.279525]  ? exit_to_usermode_loop+0x92/0xa0
[149492.279528]  entry_SYSCALL_64_fastpath+0x13/0x94
[149492.279529] RIP: 0033:0x7fc53fcb68e7
[149492.279529] RSP: 002b:00007ffcfbfaabb8 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0
[149492.279531] RAX: ffffffffffffffda RBX: 0000563117adb200 RCX: 00007fc53fcb68e7
[149492.279531] RDX: 000000000000000a RSI: 0000000000000800 RDI: 0000563117adb268
[149492.279532] RBP: 0000000000000003 R08: 0000000000000000 R09: 1999999999999999
[149492.279533] R10: 0000000000000883 R11: 0000000000000206 R12: 00007ffcfbfa9ba0
[149492.279533] R13: 0000000000000000 R14: 0000000000000000 R15: 0000563117adb200
[149492.279534] Code: 55 48 89 e5 e8 77 fe ff ff 84 c0 74 02 5d c3 80 3d 40 f2 a4 00 00 75 f5 48 c7 c7 20 3c ca 81 c6 05 30 f2 a4 00 01 e8 91 f0 d7 ff <0f> ff 5d c3 90 55 48 89 fe bf 01 00 00 00 48 89 e5 e8 9f fe ff
[149492.279557] ---[ end trace 2d4e0ffcb66a1016 ]---

Unref the fence *after* waiting for it.

v2: Set man->move to NULL after dropping the last ref (Christian König)

Fixes: aff98ba1 (drm/ttm: wait for eviction in ttm_bo_force_list_clean)
Signed-off-by: NJohn Brooks <john@fastquake.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Now that drm_[cm]alloc* helpers are simple one line wrappers around
kvmalloc_array and drm_free_large is just kvfree alias we can drop
them and replace by their native forms.

This shouldn't introduce any functional change.

Changes since v1
- fix typo in drivers/gpu//drm/etnaviv/etnaviv_gem.c - noticed by 0day
  build robot
Suggested-by: NDaniel Vetter <daniel@ffwll.ch>
Signed-off-by: Michal Hocko <mhocko@suse.com>drm: drop drm_[cm]alloc* helpers
[danvet: Fixup vgem which grew another user very recently.]
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: NChristian König <christian.koenig@amd.com>
Link: http://patchwork.freedesktop.org/patch/msgid/20170517122312.GK18247@dhcp22.suse.cz

For the C file, include <drm/*.h> instead of relative path from
include/drm.

For headers in include/drm/ttm, simplify the <tty/*.h> with "*.h".

This allows us to remove the -Iinclude/drm compiler flag from
drivers/gpu/drm/ttm/Makefile (and from other drivers' Makefiles).
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1493009447-31524-3-git-send-email-yamada.masahiro@socionext.com

set_memory_* functions have moved to set_memory.h.  Switch to this
explicitly.

[akpm@linux-foundation.org: track drivers/gpu/drm/i915/i915_gem_gtt.c linux-next changes]
Link: http://lkml.kernel.org/r/1488920133-27229-8-git-send-email-labbott@redhat.comSigned-off-by: NLaura Abbott <labbott@redhat.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The ret must be 0 here, otherwise, the function will return after init_mem_type.
Signed-off-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This allows drivers to specify if they need a contiguous allocation or not.

v2: use space instead of tab
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This allows the driver to handle io_mem mappings on their own.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

No need to implement the same logic twice. Also check if the busy placements
are identical to the already scanned placements before checking them.

v2: improve check even more as suggested by Michel.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>