This name makes a lot more sense, since these are about managing
driver resources rather than just memory ranges.
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-59-airlied@gmail.com

This converts vmwgfx over to using an interface to set the
in use and check the in use flag.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-57-airlied@gmail.com

under driver control, this flag isn't needed anymore,
remove the API that used to access it, and consoldiate
with the used api.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-56-airlied@gmail.com

The driver now controls these, the core just controls the system
memory one.

v2: init sysman explicitly and assign it as a driver manager
to simplify the lookup sequence.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-55-airlied@gmail.com

This link isn't needed anymore, drop it from the init interface.
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-54-airlied@gmail.com

Other users of this should notice this change and switch to wrapper.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-44-airlied@gmail.com

This will allow different abstractions later.
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-37-airlied@gmail.com

Drivers should all be cleaning up their memory managers
themselves now, so let the core just clean the system one up.

Remove the legacy cleaning interface.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-36-airlied@gmail.com

Allow the takedown path callback to be optional as well.

v2: use fini for range manager
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-27-airlied@gmail.com

The disable path is just temporary for now, it will be dropped once has_type
is gone in a later patch.

v2: add docs.
rename to ttm_mem_type_manager namespace
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-26-airlied@gmail.com

This makes it easier to cleanup things
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-25-airlied@gmail.com

Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-24-airlied@gmail.com

Remove the exit path, since this can't fail now.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-23-airlied@gmail.com

This will allow the driver to control the ordering here better.

Eventually the old path will be removed.

v2: add docs for new APIs.
rename new path to ttm_mem_type_manager_init/set_used(for now)
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-14-airlied@gmail.com

As suggested on review, just export the memory type debug for
drivers to use, while also making the debug callback optional
(don't need to test for system as it won't init it).

rename it to be more consistent with object name for now.
(we may rename all the objects later.)
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-9-airlied@gmail.com

The pattern was repeated a few times, just make an inline for it.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200804025632.3868079-7-airlied@gmail.com

It is a very strange concept to call a function which just
calls back the caller for the functions parameters.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NThomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/382085/

Instead use a boolean field in the memory manager structure.

Also invert the meaning of the field since the use of a TT
structure is the special case here.

v2: cleanup zero init.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NThomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Link: https://patchwork.freedesktop.org/patch/382079/

Instead of repeating that in each driver.

v2: keep the caching limitation for VMWGFX for now.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NThomas Zimmermann <tzimmermann@suse.de>
Link: https://patchwork.freedesktop.org/patch/382078/

We can't pipeline that during eviction because the memory needs
to be available immediately.

v2: fix how we cleanup the BOs resources
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Tested-by: NAlex Sierra <alex.sierra@amd.com>
Link: https://patchwork.freedesktop.org/patch/379039/

The original intention was to avoid CPU page table unmaps
when BOs move between the GTT and SYSTEM domain.

The problem is that this never correctly handled changes
in the caching attributes or backing pages.

Just drop this for now and simply unmap the CPU page
tables in all cases.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/378240/

Just use the use_io_reserve_lru flag. It doesn't make much
sense to have two flags.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/378238/

Stop touching the backend private pointer alltogether and
make sure we never put the same mem twice by.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMadhav Chauhan <madhav.chauhan@amd.com>
Link: https://patchwork.freedesktop.org/patch/375613/

We only need the page array when the BO is about to be accessed.

So not only populate, but also create it on demand.

v2: move NULL check into ttm_tt_create()
v3: fix the occurrence in ttm_bo_kmap_ttm as well
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMichael J. Ruhl <michael.j.ruhl@intel.com>
Link: https://patchwork.freedesktop.org/patch/373182/

Instead of signaling failure by setting the node pointer to
NULL do so by returning -ENOSPC.

v2: add memset() to make sure that mem is always initialized.
v3: drop memset() only set mm_node = NULL, move mm_node init in amdgpu
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMichael J. Ruhl <michael.j.ruhl@intel.com>
Link: https://patchwork.freedesktop.org/patch/373181/

GPU address handling is device specific and should be handle by its device
driver.
Signed-off-by: NNirmoy Das <nirmoy.das@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/372937/Signed-off-by: NChristian König <christian.koenig@amd.com>

Calling ttm_bo_cleanup_memtype_use() destroys the TT object
which in turn could result in warnings without this.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: NSumit Semwal <sumit.semwal@linaro.org>
Link: https://patchwork.freedesktop.org/patch/359288

The assert sometimes incorrectly triggers when pinned BOs are destroyed.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Tested-by: NPierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Link: https://patchwork.freedesktop.org/patch/356737/

Pipeline removal of the BOs backing store when no placement is given
during validation.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/353994/?series=73646&rev=1

This allows release_notify to add and remove fences from deleted objects.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Nxinhui pan <xinhui.pan@amd.com>
Link: https://patchwork.freedesktop.org/patch/352750/

When non-imported BOs are resurrected for delayed delete we replace
the dma_resv object to allow for easy reclaiming of the resources.

v2: move that to ttm_bo_individualize_resv
v3: add a comment to explain what's going on
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Nxinhui pan <xinhui.pan@amd.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/352927/

This patch reworks the whole delayed deletion of BOs which aren't idle.

Instead of having two counters for the BO structure we resurrect the BO
when we find that a deleted BO is not idle yet.

This has many advantages, especially that we don't need to
increment/decrement the BOs reference counter any more when it
moves on the LRUs.

v2: remove duplicate ttm_tt_destroy, fix holde lock for LRU move
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Nxinhui pan <xinhui.pan@amd.com>
Link: https://patchwork.freedesktop.org/patch/352912/

This allows it to call the function without the lock held.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Nxinhui pan <xinhui.pan@amd.com>
Link: https://patchwork.freedesktop.org/patch/352742/

The function is always called with deleted BOs.

While at it cleanup the indentation as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Nxinhui pan <xinhui.pan@amd.com>
Link: https://patchwork.freedesktop.org/patch/352743/

As we move the ttm_bo_individualize_resv() upwards, we need flush the
copied fence too. Otherwise the driver keeps waiting for fence.

run&Kill kfdtest, then perf top.

  25.53%  [ttm]                     [k] ttm_bo_delayed_delete
  24.29%  [kernel]                  [k] dma_resv_test_signaled_rcu
  19.72%  [kernel]                  [k] ww_mutex_lock

Fix: 378e2d5b("drm/ttm: fix ttm_bo_cleanup_refs_or_queue once more")
Signed-off-by: Nxinhui pan <xinhui.pan@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/series/72339/Signed-off-by: NChristian König <christian.koenig@amd.com>

Another completely unused feature.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Link: https://patchwork.freedesktop.org/patch/348265/

With nouveau fixed all ttm-using drives have the correct nesting of
mmap_sem vs dma_resv, and we can just lock the buffer.

Assuming I didn't screw up anything with my audit of course.

v2:
- Dont forget wu_mutex (Christian König)
- Keep the mmap_sem-less wait optimization (Thomas)
- Use _lock_interruptible to be good citizens (Thomas)

v3: Rebase over fault handler helperification.

Reviewed-by: Christian König <christian.koenig@amd.com> (v2)
Reviewed-by: Thomas Hellström <thellstrom@vmware.com> (v2)
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Cc: Christian Koenig <christian.koenig@amd.com>
Cc: Huang Rui <ray.huang@amd.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: "VMware Graphics" <linux-graphics-maintainer@vmware.com>
Cc: Thomas Hellstrom <thellstrom@vmware.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20191104173801.2972-3-daniel.vetter@ffwll.ch

As the name says global memory and bo accounting is global. So it doesn't
make to much sense having pointers to global structures all around the code.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NThomas Hellström <thellstrom@vmware.com>
Link: https://patchwork.freedesktop.org/patch/332879/

This allows blocking for BOs to become available
in the memory management.

Amdgpu is doing this for quite a while now during CS. Now
apply the new behavior to all drivers using TTM.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Acked-by: NThomas Hellstrom <thellstrom@vmware.com>
Link: https://patchwork.freedesktop.org/patch/332878/

This feature is only used by vmwgfx and superfluous for everybody else.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Co-developed-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Tested-by: NThomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Link: https://patchwork.freedesktop.org/patch/333650/