Implement the shutdown callback for adreno gpu platform device
to safely shutdown it before a system reboot. This helps to avoid
futher transactions from gpu after the smmu is moved to bypass mode.
Signed-off-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Signed-off-by: NRob Clark <robdclark@chromium.org>

Neither of these code-paths apply to older 32b devices, but it is rude
to introduce warnings.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NDave Airlie <airlied@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20200929001925.2916984-1-robdclark@gmail.com

Commit 604234f3 ("drm/msm: Enable expanded apriv support for a650")
was checking the result of adreno_is_a650() before the gpu revision
got probed in adreno_gpu_init() so it was always coming across as
false. Snoop into the revision ID ahead of time to correctly set the
hw_apriv flag so that it can be used by msm_gpu to properly setup
global buffers.

Fixes: 604234f3 ("drm/msm: Enable expanded apriv support for a650")
Reported-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Tested-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>

As newer GPU families are added it makes less sense to maintain a
"generic" version functions for older families. Move adreno_submit()
and get_rptr() into the target specific code for a2xx, a3xx and a4xx.
Add a parameter to adreno_flush to pass the target specific WPTR register
instead of relying on the generic register.

All of this gets rid of the last of the REG_ADRENO offsets so remove all
all the register definitions and infrastructure.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Support the WHERE_AM_I opcode for the A618, A630 and A640 GPUs if the
microcode supports it. The WHERE_AM_I opcode allows the RPTR shadow
to be updated in priviliged memory which protects the shadow from being
read or written from user submissions.

A650 already supports extended APRIV have built in hardware support for
to access privilged memory from the CP and can go back to using the
hardware RPTR shadow feature.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Newer microcode versions have support for the CP_WHERE_AM_I opcode which
allows the RPTR shadow memory to be marked as privileged to protect it
from corruption. Move the RPTR shadow into its own buffer and protect it
it if the current microcode version supports the new feature.

We can also re-enable preemption for those targets that support
CP_WHERE_AM_I. Start out by preemptively assuming that we can enable
preemption and disable it in a5xx_hw_init if the microcode version comes
back as too old.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Add support for using per-instance pagetables if all the dependencies are
available.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NAkhil P Oommen <akhilpo@codeaurora.org>

Use the aperture settings from the IOMMU domain to set up the virtual
address range for the GPU. This allows us to transparently deal with
IOMMU side features (like split pagetables).
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>

Now that we can get the ctx from the submitqueue, the extra arg is
redundant.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
[split out of previous patch to reduce churny noise]
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>

This will be populated by adreno-smmu, to provide a way for coordinating
enabling/disabling TTBR0 translation.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>

In a later patch, the drvdata will not directly be 'struct msm_gpu *',
so add a helper to reduce the churn.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>

It's allocating an array of a6xx_gpu_state_obj structure rathor than
its pointers.

This patch fix it.
Signed-off-by: NZhenzhong Duan <zhenzhong.duan@gmail.com>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The function iommu_domain_alloc returns NULL on platforms without IOMMU
such as msm8974. This resulted in PTR_ERR(-ENODEV) being assigned to
gpu->aspace so the correct code path wasn't taken.

Fixes: ccac7ce3 ("drm/msm: Refactor address space initialization")
Signed-off-by: NLuca Weiss <luca@z3ntu.xyz>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Signed-off-by: NRob Clark <robdclark@chromium.org>

Technically the GMU specific one is a bit redundant, but it was useful
to track down a bug.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>

XX_print like pfp_print/me_print/meq_print/roq_print are just
used in file a5xx_debugfs.c. And these function always return
0, this return value is meaningless.
This change is to make the code a bit more readable.
Signed-off-by: NBernard Zhao <bernard@vivo.com>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Disable the RPTR shadow across all targets. It will be selectively
re-enabled later for targets that need it.

Cc: stable@vger.kernel.org
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Temporarily disable preemption on a5xx targets pending some improvements
to protect the RPTR shadow from being corrupted.

Cc: stable@vger.kernel.org
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

a650 supports expanded apriv support that allows us to map critical buffers
(ringbuffer and memstore) as as privileged to protect them from corruption.

Cc: stable@vger.kernel.org
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The main a5xx preemption record can be marked as privileged to
protect it from user access but the counters storage needs to be
remain unprivileged. Split the buffers and mark the critical memory
as privileged.

Cc: stable@vger.kernel.org
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Replace the existing /* fall through */ comments and its variants with
the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary
fall-through markings when it is the case.

[1] https://www.kernel.org/doc/html/v5.7/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-throughSigned-off-by: NGustavo A. R. Silva <gustavoars@kernel.org>

The patch reorganizing the set_freq function made it so the gmu resume
doesn't always set the frequency, because a6xx_gmu_set_freq() exits early
when the frequency hasn't been changed. Note this always happens when
resuming GMU after recovering from a hang.

Use a simple workaround to prevent this from happening.

Fixes: 1f60d114 ("drm: msm: a6xx: send opp instead of a frequency")
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>

For production devices, the debugbus sections will typically be fused
off and empty in the gpu device coredump.  But since this may contain
data like cache contents, don't capture it by default.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Backport note: maybe wait some time for the crashdec MR[1] to look for
both the old typo'd name and the corrected name to land in mesa 20.2

[1] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6242

Fixes: 1707add8 ("drm/msm/a6xx: Add a6xx gpu state")
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

New Qualcomm firmware has changed a way it reports back the 'started'
event. Support new register values.
Signed-off-by: NDmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

This patches replaces the previously used static DDR vote and uses
dev_pm_opp_set_bw() to scale GPU->DDR bandwidth along with scaling
GPU frequency. Also since the icc path voting is handled completely
in the opp driver, remove the icc_path handle and its usage in the
drm driver.
Signed-off-by: NSharat Masetty <smasetty@codeaurora.org>
Signed-off-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

We need to set it to the most recent completed fence, not the most
recent submitted.  Otherwise we have races where we think we can retire
submits that the GPU is not finished with, if the GPU doesn't manage to
overwrite the seqno before we look at it.

This can show up with hang recovery if one of the submits after the
crashing submit also hangs after it is replayed.

Fixes: f97decac ("drm/msm: Support multiple ringbuffers")
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Initialize hardware clock-gating registers on A640 and A650 GPUs.

At least for A650, this solves some performance issues.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>

This will allow supporting different hwcg tables for a6xx.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>

This is required for A640 and A650 to be able to share UBWC-compressed
images with other HW such as display, which expect this configuration.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Small cleanup, lets not open-code bits/bitfields that are properly
defined in the rnndb xml (and therefore have builders in the generated
headers)
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

We haven't sync'd for a while.. pull in updates to get definitions for
some fields in pkt7 payloads.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Acked-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

This sets up bw tables for A640/A650 similar to A618/A630, 0 DDR bandwidth
vote, and the CNOC vote. A640 has the same CNOC addresses as A630 and was
working, but this is required for A650 to work.

Eventually the bw table should be filled by querying the interconnect
driver for each BW in the dts, but use these dummy tables for now.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

A650 has a separate RSCC region, so dump RSCC registers separately, reading
them from the RSCC base. Without this change a GPU hang will cause a system
reset if CONFIG_DEV_COREDUMP is enabled.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

On targets where GMU is available, GMU takes over the ownership of GX GDSC
during its initialization. So, move the refcount-get on GX PD before we
initialize the GMU. This ensures that nobody can collapse the GX GDSC
once GMU owns the GX GDSC. This patch fixes some GMU OOB errors seen
during GPU wake up during a system resume.
Reported-by: NMatthias Kaehlcke <mka@chromium.org>
Signed-off-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Tested-by: NMatthias Kaehlcke <mka@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

This patch changes the plumbing to send the devfreq recommended opp rather
than the frequency. Also consolidate and rearrange the code in a6xx to set
the GPU frequency and the icc vote in preparation for the upcoming
changes for GPU->DDR scaling votes.
Signed-off-by: NSharat Masetty <smasetty@codeaurora.org>
Signed-off-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

If there is no interconnect-names, but there is an interconnects
property, then of_icc_get(dev, "gfx-mem"); would return an error
rather than NULL.

Also, if there is no interconnect-names property, there will never
be a ocmem path.  But of_icc_get(dev, "ocmem") would return -EINVAL
instead of -ENODATA.  Just don't bother trying in this case.

v2: explicity check for interconnect-names property

Fixes: 08af4769 ("drm/msm: handle for EPROBE_DEFER for of_icc_get")
Fixes: 00bb9243 ("drm/msm/gpu: add support for ocmem interconnect path")
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Check for errors instead of silently not using icc if the msm driver
probes before the interconnect driver.

Allow ENODATA for ocmem path, as it is optional and this error
is returned when "gfx-mem" path is provided but not "ocmem".

Because msm_gpu_cleanup assumes msm_gpu_init has been called, the icc path
init needs to be after msm_gpu_init for the error path to work.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>

adreno_gpu_init calls pm_runtime_enable, so adreno_gpu_cleanup needs to
call pm_runtime_disable.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>