pm_runtime_get_sync returns negative on failure.

Fixes: eaeb9010 ("drm/nouveau/debugfs: Wake up GPU before doing any reclocking")
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

The hmm_devmem_add() function doesn't return NULL, it returns error
pointers.

Fixes: 5be73b69 ("drm/nouveau/dmem: device memory helpers for SVM")
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Fixes gcc '-Wunused-but-set-variable' warning:

drivers/gpu/drm/nouveau/nouveau_dmem.c: In function 'nouveau_dmem_free':
drivers/gpu/drm/nouveau/nouveau_dmem.c:103:22: warning:
 variable 'drm' set but not used [-Wunused-but-set-variable]
  struct nouveau_drm *drm;
                      ^
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Removes the need for temporary VMM mappings.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Removes the need for temporary VMM mappings.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This add an ioctl to migrate a range of process address space to the
device memory. On platform without cache coherent bus (x86, ARM, ...)
this means that CPU can not access that range directly, instead CPU
will fault which will migrate the memory back to system memory.

This is behind a staging flag so that we can evolve the API.
Signed-off-by: NJérôme Glisse <jglisse@redhat.com>

Device memory can be use in SVM, in which case we do not have any of
the existing buffer object. This commit add infrastructure to allow
use of device memory without nouveau_bo. Again this is a temporary
solution until a rework of GPU memory management.
Signed-off-by: NJérôme Glisse <jglisse@redhat.com>

This uses HMM to mirror a process' CPU page tables into a channel's page
tables, and keep them synchronised so that both the CPU and GPU are able
to access the same memory at the same virtual address.

While this code also supports Volta/Turing, it's only enabled for Pascal
GPUs currently due to channel recovery being unreliable right now on the
later GPUs.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

For a channel to make use of SVM features, it requires a different GPU MMU
configuration than we would normally use, which is not desirable to switch
to unless a client is actively going to use SVM.

In order to supporting SVM without more extensive changes to the userspace
interfaces, the SVM_INIT ioctl needs to replace the previous configuration
safely.

The only way we can currently do this safely, accounting for some unlikely
failure conditions, is to allocate the new VMM without destroying the last
one, and prioritising the SVM-enabled configuration in the code that cares.

This will get cleaned up again further down the track.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This nvclass exposes the replayable fault buffer, which will be used
by SVM to manage GPU page faults.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This nvclass exposes the replayable fault buffer, which will be used
by SVM to manage GPU page faults.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Some GPU units are capable of supporting "replayable" page faults, where
the execution unit will wait for SW to fixup GPU page tables rather than
triggering a channel-fatal fault.

This feature isn't useful (it's harmful, even) unless something like HMM
is being used to manage events appearing in the replayable fault buffer,
so, it's disabled by default.

This commit allows a client to request it be enabled.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Host methods exist to do at least some of what we need, but we are not
currently pushing replay/cancels through a channel like UVM does as it's
not clear whether it's necessary in our case (UVM also updates PTEs with
the GPU).

UVM also pushes a software method for fault cancels on Pascal, seemingly
because the host methods don't appear to be sufficient.  If/when we want
to push the replay/cancel on the GPU, we can re-purpose the cancellation
code here to implement that swmthd.

Keep it simple for now, until we figure out exactly what we need here.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This provides a somewhat more direct method of manipulating the GPU page
tables, which will be required to support SVM.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This will be used to support a privileged client providing PTEs directly,
without a memory object to use as a reference.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

NVKM is currently responsible for managing the allocation of a client's
GPU address-space, but there's various use-cases (ie. HMM address-space
mirroring) where giving a client more direct control is desirable.

This commit allows for a VMM to be created where the area allocated for
NVKM is limited to a client-specified window, the remainder of address-
space is controlled directly by the client.

Leaving a window is necessary to support various internal requirements,
but also to support existing allocation interfaces as not all of the HW
is capable of working with a HMM allocation.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

MMU will need access to this info.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

MMU will need access to these.

v2. Apply fix from Rhys Kidd to send correct FECS method for STOP_CTXSW.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There are a few statements that are indented incorrectly. Fix these.
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

It appears that Pascal and newer need something different.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Will want to reuse this for fault replay/cancellation swmthds.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Future changes will want to add some additional things here, keep them
grouped together.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This is already done during golden context creation.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There's no need to avoid using copy engines if gr init fails for some
reason (usually missing FW, or incomplete bring-up).

It's not terribly useful for an end-user, but it'll slightly speed up
suspend/resume when saving fb contents, and allow for host/ce code to
be validated.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

It has no relevance to the atomic path used by newer GPUs.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

It has no relevance to the atomic path used by newer GPUs.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Swapped order of functions in dispnv04 to allow this, but no code changes.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Some of the pre-NV50 depends on SW methods to implement synchronisation
for page flips, and we want to move this setup out of common code, thus
we require the channel to have been allocation before display init.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

We also change the error strings to match NVIDIA's naming.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

As I currently understand it, this is related to features we have no
support for as of yet.

In theory, this change should be a noop, just without the warning.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>