Add a "best_match" flag similar to the drm_mm_search_*() helpers so we
can convert TTM to use them in follow up patches. We can also inline the
non-generic helpers and move them into the header to allow compile-time
optimizations.

To make calls to drm_mm_{search,insert}_node() more readable, this
converts the boolean argument to a flagset. There are pending patches that
add additional flags for top-down allocators and more.

v2:
 - use flag parameter instead of boolean "best_match"
 - convert *_search_free() helpers to also use flags argument
Signed-off-by: NDavid Herrmann <dh.herrmann@gmail.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

We need BUG_ON(), spinlock_t and standard kernel data-types so include the
right headers.

Subject: [drm-intel:drm-intel-nightly 154/166] include/drm/drm_mm.h:67:2:
 error: unknown type name 'spinlock_t'
Message-ID: <51f14693.g5HGdcuw2v3m8FOd%fengguang.wu@intel.com>

In case it didn't link to it correctly. Somehow this bug doesn't occur
here on my machine, hmm. But I think fixing drm_mm.h is better than
changing the include-order in drm_vma_manager.h, so this is what I
did.
Signed-off-by: NDavid Herrmann <dh.herrmann@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

With the previous patch we no longer actually create a node, we simply
find the correct hole and occupy it. This very well could have been
squashed with the last patch, but since I already had David's review, I
figured it's easiest to keep it distinct.

Also update the users in i915. Conveniently this is the only user of the
interface.

CC: David Airlie <airlied@linux.ie>
CC: <dri-devel@lists.freedesktop.org>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Acked-by: NDavid Airlie <airlied@linux.ie>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

For an upcoming patch where we introduce the i915 VMA, it's ideal to
have the drm_mm_node as part of the VMA struct (ie. it's pre-allocated).
Part of the conversion to VMAs is to kill off obj->gtt_space. Doing this
will break a bunch of code, but amongst them are 2 callers of
drm_mm_create_block(), both related to stolen memory.

It also allows us to embed the drm_mm_node into the object currently
which provides a nice transition over to the new code.

v2: Reordered to do before ripping out obj->gtt_offset.
Some minor cleanups made available because of reordering.

v3: s/continue/break on failed stolen node allocation (David)
Set obj->gtt_space on failed node allocation (David)
Only unref stolen (fix double free) on failed create_stolen (David)
Free node, and NULL it in failed create_stolen (David)
Add back accidentally removed newline (David)

CC: <dri-devel@lists.freedesktop.org>
Reviewed-by: NDavid Herrmann <dh.herrmann@gmail.com>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Acked-by: NDavid Airlie <airlied@linux.ie>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

drm/i915 is the only user of the color allocation handling and
switched to insert_node a while ago. So we can ditch this.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDave Airlie <airlied@redhat.com>

There is no reason to return "int" as this function never fails.
Furthermore, several drivers (ast, sis) already depend on this.
Signed-off-by: NDavid Herrmann <dh.herrmann@gmail.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Avoid clobbering adjacent blocks if they happen to expire earlier and
amalgamate together to form the requested hole.

In passing this fixes a regression from
commit ea7b1dd4
Author: Daniel Vetter <daniel.vetter@ffwll.ch>
Date:   Fri Feb 18 17:59:12 2011 +0100

    drm: mm: track free areas implicitly

which swaps the end address for size (with a potential overflow) and
effectively causes the eviction code to clobber almost all earlier
buffers above the evictee.

v2: Check the original hole not the adjusted as the coloring may confuse
us when later searching for the overlapping nodes. Also make sure that
we do apply the range restriction and color adjustment in the same
order for both scanning, searching and insertion.

v3: Send the version that was actually tested.

Note that this seems to be ducttape of decent quality ot paper over
some of our unbind related gpu hangs reported since 3.7. It is not
fully effective though, and certainly doesn't fix the underlying bug.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
[danvet: Added note plus bugzilla link and tested-by.]
Cc: stable@vger.kernel.org
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=55984Tested-by: NNorbert Preining <preining@logic.at>
Acked-by: Dave Airlie <airlied@gmail.com
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Required by i915 in order to avoid the allocation in the middle of
manipulating the drm_mm lists.

Use a pair of stubs to preserve the existing EXPORT_SYMBOLs for
backporting; to be removed later.

Cc: Dave Airlie <airlied@redhat.com>
Cc: dri-devel@lists.freedesktop.org
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
[danvet: bikeshedded-away the atomic parameter, it's not yet used
anywhere.]
Acked-by: NDave Airlie <airlied@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

This will be used i915 in forthcoming patches in order to measure the
largest contiguous chunk of memory available for enabling chipset
features.

v2: Try to make the macro marginally safer and more readable by not
depending upon the drm_mm_hole_node_end() being non-zero. Note that we
need to open code list_for_each() in order to update the hole_start,
hole_end variable on each iteration and keep the macro sane.

v3: Tidy up few BUG_ONs that fell foul of adding additional tests to
drm_mm_hole_node_start().
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Dave Airlie <airlied@redhat.com>
Acked-by: NDave Airlie <airlied@redhat.com>
Cc: dri-devel@lists.freedesktop.org
Reviewed-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

To be used later by i915 to preallocate exact blocks of space from the
range manager.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Dave Airlie <airlied@redhat.com>
Acked-by: NDave Airlie <airlied@redhat.com>
Cc: dri-devel@lists.freedesktop.org
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In order to support snoopable memory on non-LLC architectures (so that
we can bind vgem objects into the i915 GATT for example), we have to
avoid the prefetcher on the GPU from crossing memory domains and so
prevent allocation of a snoopable PTE immediately following an uncached
PTE. To do that, we need to extend the range allocator with support for
tracking and segregating different node colours.

This will be used by i915 to segregate memory domains within the GTT.

v2: Now with more drm_mm helpers and less driver interference.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Dave Airlie <airlied@redhat.com
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Ben Skeggs <bskeggs@redhat.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@gmail.com>

The looping helper didn't do anything due to a superficial
semicolon. Furthermore one of the two dump functions suffered
from copy&paste fail.

While staring at the code I've also noticed that the replace
helper (currently unused) is a bit broken.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Fixes generated by 'codespell' and manually reviewed.
Signed-off-by: NLucas De Marchi <lucas.demarchi@profusion.mobi>

With the switch to implicit free space accounting one pointer
got unused when scanning. Use it to create a single-linked list
to ensure correct unwinding of the scan state.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

The old api has a two-step process: First search for a suitable
free hole, then allocate from that specific hole. No user used
this to do anything clever. So drop it for the embeddable variant
of the drm_mm api (the old one retains this ability, for the time
being).

With struct drm_mm_node embedded, we cannot track allocations
anymore by checking for a NULL pointer. So keep track of this
and add a small helper drm_mm_node_allocated.

Also add a function to move allocations between different struct
drm_mm_node.

v2: Implement suggestions by Chris Wilson.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

The idea is to track free holes implicitly by marking the allocation
immediatly preceeding a hole.

To avoid an ugly corner case add a dummy head_node to struct drm_mm
to track the hole that spans to complete allocation area when the
memory manager is empty.

To guarantee that there's always a preceeding/following node (that might
be marked as hole_follows == 1), move the mm->node_list list_head to the
head_node.

The main allocator and fair-lru scan code actually becomes simpler.
Only the debug code slightly suffers because free areas are no longer
explicit.

Also add drm_mm_for_each_node (which will be much more useful when
struct drm_mm_node is embeddable).
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Nouveau was checking drm_mm internals on teardown to see whether the
memory manager was initialized. Hide these internals in a small
inline helper function.
Acked-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>

These helper functions can be used to efficiently scan lru list
for eviction. Eviction becomes a three stage process:
1. Scanning through the lru list until a suitable hole has been found.
2. Scan backwards to restore drm_mm consistency and find out which
   objects fall into the hole.
3. Evict the objects that fall into the hole.

These helper functions don't allocate any memory (at the price of
not allowing any other concurrent operations). Hence this can also be
used for ttm (which does lru scanning under a spinlock).

Evicting objects in this fashion should be more fair than the current
approach by i915 (scan the lru for a object large enough to contain
the new object). It's also more efficient than the current approach used
by ttm (uncoditionally evict objects from the lru until there's enough
free space).
Signed-Off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: NThomas Hellstrom <thellstrom@vmwgfx.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Yeah, I've kinda noticed that fl_entry is the free stack. Still
give it (and the memory node list ml_entry) decent names.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: NThomas Hellstrom <thellstrom@vmwgfx.com>
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Only ever assigned, never used.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
[glisse: I will re-add if needed for range-restricted allocations]
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NDave Airlie <airlied@redhat.com>

drm_mm_debug_table will print the memory manager state
in table allowing to give a snapshot of the manager at
given point in time. Usefull for debugging.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

These are required for changes to TTM.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Fixes a warning seen on powerpc.
Signed-off-by: NDave Airlie <airlied@redhat.com>

This adds code to the drm_mm to talk to debugfs, and adds
support to radeon to add the VRAM and GTT mm lists to debugfs.

I tested with spinlock debugging and it doesn't give out.
Signed-off-by: NDave Airlie <airlied@redhat.com>

also for the atomic path by using a common code-path.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

this is a TTM preparation patch, it rearranges the mm and
add operations needed to do mm operations in atomic context.
Signed-off-by: NThomas Hellstrom <thellstrom@vmware.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>