This was noticed by Gustavo and his -Wimplicit-fallthrough
patches.  However, in this case, I believe we should have breaks
rather than falling though, that said, in practice we should
never fall through in the first place so there should be no
change in behavior.
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Simplify the ppfeature mask calculations.
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NKenneth Feng <kenneth.feng@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When DPM for the specific clock is disabled, driver should still print out
current clock info for rocm-smi support on vega20
Signed-off-by: Nshaoyunl <shaoyun.liu@amd.com>
Reviewed-by: NEric Huang <JinhuiEric.Huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Fixes for 5.1:
amdgpu:
- Fix missing fw declaration after dropping old CI DPM code
- Fix debugfs access to registers beyond the MMIO bar size
- Fix context priority handling
- Add missing license on some new files
- Various cleanups and bug fixes

radeon:
- Fix missing break in CS parser for evergreen
- Various cleanups and bug fixes

sched:
- Fix entities with 0 run queues
Signed-off-by: NDave Airlie <airlied@redhat.com>
From: Alex Deucher <alexdeucher@gmail.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190221214134.3308-1-alexander.deucher@amd.com

Signed-off-by: NBas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

s/BOCO/BACO/g
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Use a proper return code rather than -1.
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Trivial.
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

On the display side, cleanups and fixes to enabled modifiers
(QCOM_COMPRESSED).  And otherwise mostly misc fixes all around.
Signed-off-by: NDave Airlie <airlied@redhat.com>
From: Rob Clark <robdclark@gmail.com>
Link: https://patchwork.freedesktop.org/patch/msgid/CAF6AEGuZ5uBKpf=fHvKpTiD10nychuEY8rnE+HeRz0QMvtY5_A@mail.gmail.com

Various fixes/cleanups, along with initial support for SVM features
utilising HMM address-space mirroring and device memory migration.
There's a lot more work to do in these areas, both in terms of
features and efficiency, but these can slowly trickle in later down
the track.
Signed-off-by: NDave Airlie <airlied@redhat.com>
From: Ben Skeggs <skeggsb@gmail.com>
Link: https://patchwork.freedesktop.org/patch/msgid/CACAvsv5bsB4rRY1Gqa_Bp_KAd-v_q1rGZ4nYmOAQhceL0Nr-Xg@mail.gmail.com

Removes the need for temporary VMM mappings.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Removes the need for temporary VMM mappings.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This add an ioctl to migrate a range of process address space to the
device memory. On platform without cache coherent bus (x86, ARM, ...)
this means that CPU can not access that range directly, instead CPU
will fault which will migrate the memory back to system memory.

This is behind a staging flag so that we can evolve the API.
Signed-off-by: NJérôme Glisse <jglisse@redhat.com>

Device memory can be use in SVM, in which case we do not have any of
the existing buffer object. This commit add infrastructure to allow
use of device memory without nouveau_bo. Again this is a temporary
solution until a rework of GPU memory management.
Signed-off-by: NJérôme Glisse <jglisse@redhat.com>

This uses HMM to mirror a process' CPU page tables into a channel's page
tables, and keep them synchronised so that both the CPU and GPU are able
to access the same memory at the same virtual address.

While this code also supports Volta/Turing, it's only enabled for Pascal
GPUs currently due to channel recovery being unreliable right now on the
later GPUs.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

For a channel to make use of SVM features, it requires a different GPU MMU
configuration than we would normally use, which is not desirable to switch
to unless a client is actively going to use SVM.

In order to supporting SVM without more extensive changes to the userspace
interfaces, the SVM_INIT ioctl needs to replace the previous configuration
safely.

The only way we can currently do this safely, accounting for some unlikely
failure conditions, is to allocate the new VMM without destroying the last
one, and prioritising the SVM-enabled configuration in the code that cares.

This will get cleaned up again further down the track.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This nvclass exposes the replayable fault buffer, which will be used
by SVM to manage GPU page faults.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This nvclass exposes the replayable fault buffer, which will be used
by SVM to manage GPU page faults.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Some GPU units are capable of supporting "replayable" page faults, where
the execution unit will wait for SW to fixup GPU page tables rather than
triggering a channel-fatal fault.

This feature isn't useful (it's harmful, even) unless something like HMM
is being used to manage events appearing in the replayable fault buffer,
so, it's disabled by default.

This commit allows a client to request it be enabled.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Host methods exist to do at least some of what we need, but we are not
currently pushing replay/cancels through a channel like UVM does as it's
not clear whether it's necessary in our case (UVM also updates PTEs with
the GPU).

UVM also pushes a software method for fault cancels on Pascal, seemingly
because the host methods don't appear to be sufficient.  If/when we want
to push the replay/cancel on the GPU, we can re-purpose the cancellation
code here to implement that swmthd.

Keep it simple for now, until we figure out exactly what we need here.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This provides a somewhat more direct method of manipulating the GPU page
tables, which will be required to support SVM.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This will be used to support a privileged client providing PTEs directly,
without a memory object to use as a reference.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

NVKM is currently responsible for managing the allocation of a client's
GPU address-space, but there's various use-cases (ie. HMM address-space
mirroring) where giving a client more direct control is desirable.

This commit allows for a VMM to be created where the area allocated for
NVKM is limited to a client-specified window, the remainder of address-
space is controlled directly by the client.

Leaving a window is necessary to support various internal requirements,
but also to support existing allocation interfaces as not all of the HW
is capable of working with a HMM allocation.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

MMU will need access to this info.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

MMU will need access to these.

v2. Apply fix from Rhys Kidd to send correct FECS method for STOP_CTXSW.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There are a few statements that are indented incorrectly. Fix these.
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

It appears that Pascal and newer need something different.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Will want to reuse this for fault replay/cancellation swmthds.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Future changes will want to add some additional things here, keep them
grouped together.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This is already done during golden context creation.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There's no need to avoid using copy engines if gr init fails for some
reason (usually missing FW, or incomplete bring-up).

It's not terribly useful for an end-user, but it'll slightly speed up
suspend/resume when saving fb contents, and allow for host/ce code to
be validated.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

It has no relevance to the atomic path used by newer GPUs.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

It has no relevance to the atomic path used by newer GPUs.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>