Clean up the enable sequence as well.

V2: clean up duplicate defines
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: Don't forget git add, noticed by David.

Cc: David Herrmann <dh.herrmann@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Acked-by: NDavid Herrmann <dh.herrmann@gmail.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This allows us to specify if we want to sync to
the shared fences of a reservation object or not.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

DRM_DEBUG_CODE is currently always set, so distributions enable it. The
only reason to keep support in code is if developers wanted to disable
debug support. Sounds unlikely.

All the DRM_DEBUG() printks are still guarded by a drm_debug read. So if
its cacheline is read once, they're discarded pretty fast.. There should
hardly be any performance penalty, it's even guarded by unlikely().
Signed-off-by: NDavid Herrmann <dh.herrmann@gmail.com>
Reviewed-by: NThierry Reding <treding@nvidia.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Changes since v1:
- Kill the sw interrupt dance, add and use
  radeon_irq_kms_sw_irq_get_delayed instead.
- Change custom wait function, lockdep complained about it.
  Holding exclusive_lock in the wait function might cause deadlocks.
  Instead do all the processing in .enable_signaling, and wait
  on the global fence_queue to pick up gpu resets.
- Process all fences in radeon_gpu_reset after reset to close a race
  with the trylock in enable_signaling.
Changes since v2:
- Small changes to work with the rewritten lockup recovery patches.
Changes since v3:
- Call radeon_fence_schedule_check when exclusive_lock cannot be
  acquired to always cause a wake up.
- Reset irqs from hangup check.
- Drop reading seqno in the callback, use cached value.
- Fix indentation in radeon_fence_default_wait
- Add a radeon_test_signaled function, drop a few test_bit calls.
- Make to_radeon_fence global.
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

This improves concurrent stream decoding.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Blocking completely innocent processes with a GPU reset is
a pretty bad idea. Just set needs_reset and let the next
command submission or fence wait do the job.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It's causing issues with VMID handling and comparing the
fence value two times actually doesn't make handling faster.

v2: rebased on reset changes
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v5 (chk): complete rework, start when the first fence is emitted,
          stop when the last fence is signalled, make it work
          correctly with GPU resets, cleanup radeon_fence_wait_seq
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is needed for the next commit, because the lockup detection
will need the read lock to run.

v4 (chk): split out forced fence completion, remove unrelated changes,
          add and handle in_reset flag
v5 (agd5f): rebase fix
Signed-off-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of resetting all fence numbers, only reset the
number of the problematic ring. Split out from a patch
from Maarten Lankhorst <maarten.lankhorst@canonical.com>

v2 (agd5f): rebase build fix
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMaarten Lankhorst <maarten.lankhorst@canonical.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This allows us to more fine grained specify where to place the buffer object.

v2: rebased on drm-next, add bochs changes as well
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

This fixes a problem with GPU resets and TLB flushes on SI/CIK.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It isn't necessary for command streams generated by the kernel (at least
not while we aren't storing ring or indirect buffers in VRAM).
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add a module paramter to enable bapm on APUs.  It's disabled
by default on certain APUs due to stability issues.  This
option makes it easier to test and to enable it on systems that
are stable.

bug:
https://bugzilla.kernel.org/show_bug.cgi?id=81021Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Whenever userspace mapping related to our userptr change
we wait for it to become idle and unmap it from GTT.

v2: rebased, fix mutex unlock in error path
v3: improve commit message
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This patch adds an IOCTL for turning a pointer supplied by
userspace into a buffer object.

It imposes several restrictions upon the memory being mapped:

1. It must be page aligned (both start/end addresses, i.e ptr and size).

2. It must be normal system memory, not a pointer into another map of IO
space (e.g. it must not be a GTT mmapping of another object).

3. The BO is mapped into GTT, so the maximum amount of memory mapped at
all times is still the GTT limit.

4. The BO is only mapped readonly for now, so no write support.

5. List of backing pages is only acquired once, so they represent a
snapshot of the first use.

Exporting and sharing as well as mapping of buffer objects created by
this function is forbidden and results in an -EPERM.

v2: squash all previous changes into first public version
v3: fix tabs, map readonly, don't use MM callback any more
v4: set TTM_PAGE_FLAG_SG so that TTM never messes with the pages,
    pin/unpin pages on bind/unbind instead of populate/unpopulate
v5: rebased on 3.17-wip, IOCTL renamed to userptr, reject any unknown
    flags, better handle READONLY flag, improve permission check
v6: fix ptr cast warning, use set_page_dirty/mark_page_accessed on unpin
v7: add warning about it's availability in the API definition
v8: drop access_ok check, fix VM mapping bits
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v4)
Reviewed-by: Jérôme Glisse <jglisse@redhat.com> (v4)
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Skip the "manual" pageflip completion checks via polling and
guessing in the vblank handler radeon_crtc_handle_vblank() on
asics which are known to reliably support hw pageflip completion
irqs. Those pflip irqs are a more reliable and race-free method
of handling pageflip completion detection, whereas the "classic"
polling method has some small races in combination with dpm on,
and with the reworked pageflip implementation since Linux 3.16.

On old asics without pflip irqs, the classic method is used.

On asics with known good pflip irqs, only pflip irqs are used
by default, but a new module parameter "use_pflipirqs" allows to
override this in case we encounter asics in the wild with
unreliable or faulty pflip irqs. A module parameter of 0 allows
to use the classic method only in such a case. A parameter of 1
allows to use both classic method and pflip irqs as additional
band-aid to avoid some small races which could happen with the
classic method alone. The setting 1 gives Linux 3.16 behaviour.

Hw pflip irqs are available since R600.

Tested on DCE-4, AMD Cedar - FirePro 2270.

v2:  agd5f: only enable pflip interrupts on DCE4+ as they are not
reliable on older asics.
Signed-off-by: NMario Kleiner <mario.kleiner.de@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Move the decision what to use into the common VM code.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Scales much better than scanning the address range linearly.

v2: store pfn instead of address
Signed-off-by: NChristian König <christian.koenig@amd.com>
Tested-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Don't wait for the BO to be used again, just
update the PT on the next VM use.

v2: remove stray semicolon.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Tested-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Some hawaii boards use a different method for fetching the
voltage information from the vbios.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

This ensures the GPU sees all previous CPU writes to VRAM, which makes it
safe:

* For userspace to stream data from CPU to GPU via VRAM instead of GTT
* For IBs to be stored in VRAM instead of GTT
* For ring buffers to be stored in VRAM instead of GTT, if the HPD flush
  is performed via MMIO
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

And clean up the function comment a little.
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

That didn't worked correctly any more and opened up a security problem.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Unused and unimplemented. Also fix specifying the
kernel flag incorrectly at one occasion.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Some PX laptops seems to have problems turning the dGPU on/off.
Add a quirk list to disable runpm by default on those systems.
Also convert the current PX d3 delay handling to a quirk.

bug:
https://bugzilla.kernel.org/show_bug.cgi?id=51381
https://bugzilla.kernel.org/show_bug.cgi?id=74551Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Now that fallback to gtt is fixed for cpu access, we can
remove this limit.

bug:
https://bugs.freedesktop.org/show_bug.cgi?id=78717

v2: use new gart_pin_size to accurately track available gtt.
v3: fix comment
v4: clarify comment
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>

So we know how large an allocation we can allow.

v2: incorporate Michel's comments
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>

v2: fix rebase onto drm-fixes
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Doesn't seem necessary, the GART table memory should be persistent.
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This was originally un-inlined by Andi Kleen in 2011 citing size concerns.
Indeed, a first attempt at inlining it grew radeon.ko by 7%.

However, 2% of cpu is spent in this function. Simply inlining it gave 1% more fps
in Urban Terror.

v2: We know the minimum MMIO size. Adding it to the if allows the compiler to
optimize the branch out, improving both performance and size.

The v2 patch decreases radeon.ko size by 2%. I didn't re-benchmark, but common sense
says perf is now more than 1% better.

v3: Also change _wreg, make the threshold a define.

Inlining _wreg increased the size a bit compared to v2, so now radeon.ko
is only 1% smaller.
Signed-off-by: NLauri Kasanen <cand@gmx.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This adds CIK support for the new ucode format.

v2: add size validation, integrate debug info
v3: add support for MEC2 on KV
v4: fix typos
v4: update to latest format
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This adds SI support for the new ucode format.

v2: add size validation, integrate debug info
v3: update to latest version
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Some newer boards use SVI2 for voltage control rather
than GPIO.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Calling radeon_vm_bo_find on the IB BO during CS
is illegal and can lead to an crash.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v3: completely rewritten. We now just remember which areas
    of the PT to clear and do so on the next command submission.

Bug: https://bugs.freedesktop.org/show_bug.cgi?id=79980Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

As well as enabling the vblank interrupt. These shouldn't take any
significant amount of time, but at least pinning the BO has actually been
seen to fail in practice before, in which case we need to let userspace
know about it.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: agd5f: compile fix
Signed-off-by: NStefan Brüns <stefan.bruens@rwth-aachen.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>