All available subchannels are now available for userspace to do with as it
pleases on NVC0+.

On all earlier chipsets, the kernel still uses a software object on subc 0
to implement the page flip completion method.  I hope to find some decent
way of addressing this too, but it's a tad tricker prior to fermi.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Tested-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Fence lock needs to be initialized before any call to nouveau_channel_put
because it calls nouveau_channel_idle->nouveau_fence_update which uses
fence lock.

BUG: spinlock bad magic on CPU#0, test/24134
 lock: ffff88019f90dba8, .magic: 00000000, .owner: <none>/-1, .owner_cpu: 0
Pid: 24134, comm: test Not tainted 3.0.0-nv+ #800
Call Trace:
 spin_bug+0x9c/0xa3
 do_raw_spin_lock+0x29/0x13c
 _raw_spin_lock+0x1e/0x22
 nouveau_fence_update+0x2d/0xf1
 nouveau_channel_idle+0x22/0xa0
 nouveau_channel_put_unlocked+0x84/0x1bd
 nouveau_channel_put+0x20/0x24
 nouveau_channel_alloc+0x4ec/0x585
 nouveau_ioctl_fifo_alloc+0x50/0x130
 drm_ioctl+0x289/0x361
 do_vfs_ioctl+0x4dd/0x52c
 sys_ioctl+0x42/0x65
 system_call_fastpath+0x16/0x1b

It's easily triggerable from userspace.

Additionally remove double initialization of chan->fence.pending.
Signed-off-by: NMarcin Slusarz <marcin.slusarz@gmail.com>
Cc: stable@kernel.org
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

On >=nv50, userspace would still end up allocating pushbufs in GART.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

gpuobj really needs splitting into channel/gpuobj code instead...
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Removes the need to disable IRQs to lookup channel struct on every pushbuf
ioctl, among others.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

The nouveau_wait_for_idle() call should hopefully not have been actually
necessary, we *do* wait for the channel to go idle already.  If it's
an issue somehow, the chipset-specific hooks can wait for idle themselves
before taking the lock.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This needs a massive cleanup, but to catch bugs from the interface changes
vs the engine code cleanup, this will be done later.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There's lots of more-or-less independant engines present on NVIDIA GPUs
these days, and we generally want to perform the same operations on them.
Implementing new ones requires hooking into lots of different places,
the aim of this work is to make this simpler and cleaner.

NV84:NV98 PCRYPT moved over as a test.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Fixes generated by 'codespell' and manually reviewed.
Signed-off-by: NLucas De Marchi <lucas.demarchi@profusion.mobi>

'mappable' isn't really used at all, nor is it necessary anymore as the
bo code is capable of moving buffers to mappable vram as required.

'no_vm' isn't necessary anymore either, any places that don't want to be
mapped into a GPU address space should allocate the VRAM directly instead.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Current 3D driver expects this behaviour.  While this could be changed,
there's no compelling reason to reserve more than one subchannel for the
DRM.  If we ever need to use an object other then M2MF, we can just
re-bind subchannel 0 as required.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

When hacking the libdrm for improvements, I triggered a kernel crash
related to the fact that the NOUVEAU_NOTIFIEROBJ_ALLOC ioctl calls
nouveau_channel_get with an unchecked channel index.
The patch ensures that the channel index is an unsigned and validates
its value in nouveau_channel_get.
Signed-off-by: NMichel Hermier <hermier@frugalware.org>
Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

As of this commit, it's guaranteed that if an object is in VRAM that its
GPU virtual address will be constant.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

The regs belong to PFIFO, they're different for pretty much the same
generations we need different PFIFO control for, and NVC0 is going
to be even more different than the rest.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There have been reports of PFIFO cache errors during context take down
(fdo bug 31637). They are caused by some GPU objects being taken out
while the channel is still potentially processing commands. Make sure
that all the previous rendering has landed before releasing a GPU
object.
Reported-by: NGrzesiek Sójka <pld@pfu.pl>
Reported-by: NPatrice Mandin <patmandin@gmail.com>
Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Acked-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Reviewed-by: NFrancisco Jerez <currojerez@riseup.net>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

nv0x-nv4x should be mostly fine, nv50 doesn't work yet.
Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

nouveau_fence_* functions are not type safe, which could lead to bugs.
Additionally every use of nouveau_fence_unref had to cast struct
nouveau_fence to void **.
Fix it by renaming old functions and creating static inline functions with
new prototypes. We still need old functions, because we pass function
pointers to ttm.
As we are wrapping functions, drop unused "void *arg" parameter where possible.
Signed-off-by: NMarcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This really needs cleaning up somehow, and probably investigate what's
needed to do this on earlier generations.  NVIDIA do something similar
there too.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

nouveau_channel_ref() takes a "weak" channel reference that doesn't
prevent the hardware channel resources from being released, it just
keeps the channel data structure alive.
Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

nouveau_channel_put() can be executed after the 'refcount == 0' check
in nouveau_channel_get() and before the channel reference count is
incremented. In that case CPU0 will take the context down while CPU1
thinks it owns the channel and 'refcount == 1'.
Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

The destroy_context() engine hooks call gpuobj management functions to
release the channel resources, these functions use HARDIRQ-unsafe locks
whereas destroy_context() is called with the HARDIRQ-safe
context_switch_lock held, that's a lock ordering violation.

Push the engine-specific channel destruction logic into destroy_context()
and let the hardware-specific code lock and unlock when it's actually
needed. Change the engine destruction order to avoid a race in the small
gap between pgraph and pfifo context uninitialization.
Reported-by: NMarcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This fixes a race condition between fbcon acceleration and TTM buffer
moves.  To reproduce:

- start X
- switch to vt and "while (true); do dmesg; done"
- switch to another vt and "sleep 2 && cat /path/to/debugfs/dri/0/evict_vram"
- switch back to vt running dmesg

We don't make use of this on any other channel yet, they're currently
protected by drm_global_mutex.  This will change in the near future.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Nouveau will need this on GeForce 8 and up to account for the GPU
reordering physical VRAM for some memory types.
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Acked-by: NThomas Hellström <thellstrom@vmware.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Mainly to make room for inter-channel sync.
Signed-off-by: NFrancisco Jerez <currojerez@riseup.net>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>