This will be populated by adreno-smmu, to provide a way for coordinating
enabling/disabling TTBR0 translation.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>

In a later patch, the drvdata will not directly be 'struct msm_gpu *',
so add a helper to reduce the churn.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>

It's allocating an array of a6xx_gpu_state_obj structure rathor than
its pointers.

This patch fix it.
Signed-off-by: NZhenzhong Duan <zhenzhong.duan@gmail.com>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The function iommu_domain_alloc returns NULL on platforms without IOMMU
such as msm8974. This resulted in PTR_ERR(-ENODEV) being assigned to
gpu->aspace so the correct code path wasn't taken.

Fixes: ccac7ce3 ("drm/msm: Refactor address space initialization")
Signed-off-by: NLuca Weiss <luca@z3ntu.xyz>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Signed-off-by: NRob Clark <robdclark@chromium.org>

Technically the GMU specific one is a bit redundant, but it was useful
to track down a bug.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>

XX_print like pfp_print/me_print/meq_print/roq_print are just
used in file a5xx_debugfs.c. And these function always return
0, this return value is meaningless.
This change is to make the code a bit more readable.
Signed-off-by: NBernard Zhao <bernard@vivo.com>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Disable the RPTR shadow across all targets. It will be selectively
re-enabled later for targets that need it.

Cc: stable@vger.kernel.org
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Temporarily disable preemption on a5xx targets pending some improvements
to protect the RPTR shadow from being corrupted.

Cc: stable@vger.kernel.org
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

a650 supports expanded apriv support that allows us to map critical buffers
(ringbuffer and memstore) as as privileged to protect them from corruption.

Cc: stable@vger.kernel.org
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The main a5xx preemption record can be marked as privileged to
protect it from user access but the counters storage needs to be
remain unprivileged. Split the buffers and mark the critical memory
as privileged.

Cc: stable@vger.kernel.org
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The patch reorganizing the set_freq function made it so the gmu resume
doesn't always set the frequency, because a6xx_gmu_set_freq() exits early
when the frequency hasn't been changed. Note this always happens when
resuming GMU after recovering from a hang.

Use a simple workaround to prevent this from happening.

Fixes: 1f60d114 ("drm: msm: a6xx: send opp instead of a frequency")
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>

For production devices, the debugbus sections will typically be fused
off and empty in the gpu device coredump.  But since this may contain
data like cache contents, don't capture it by default.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Backport note: maybe wait some time for the crashdec MR[1] to look for
both the old typo'd name and the corrected name to land in mesa 20.2

[1] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6242

Fixes: 1707add8 ("drm/msm/a6xx: Add a6xx gpu state")
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

New Qualcomm firmware has changed a way it reports back the 'started'
event. Support new register values.
Signed-off-by: NDmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

This patches replaces the previously used static DDR vote and uses
dev_pm_opp_set_bw() to scale GPU->DDR bandwidth along with scaling
GPU frequency. Also since the icc path voting is handled completely
in the opp driver, remove the icc_path handle and its usage in the
drm driver.
Signed-off-by: NSharat Masetty <smasetty@codeaurora.org>
Signed-off-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

We need to set it to the most recent completed fence, not the most
recent submitted.  Otherwise we have races where we think we can retire
submits that the GPU is not finished with, if the GPU doesn't manage to
overwrite the seqno before we look at it.

This can show up with hang recovery if one of the submits after the
crashing submit also hangs after it is replayed.

Fixes: f97decac ("drm/msm: Support multiple ringbuffers")
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Initialize hardware clock-gating registers on A640 and A650 GPUs.

At least for A650, this solves some performance issues.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>

This will allow supporting different hwcg tables for a6xx.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>

This is required for A640 and A650 to be able to share UBWC-compressed
images with other HW such as display, which expect this configuration.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Small cleanup, lets not open-code bits/bitfields that are properly
defined in the rnndb xml (and therefore have builders in the generated
headers)
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

We haven't sync'd for a while.. pull in updates to get definitions for
some fields in pkt7 payloads.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Acked-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

This sets up bw tables for A640/A650 similar to A618/A630, 0 DDR bandwidth
vote, and the CNOC vote. A640 has the same CNOC addresses as A630 and was
working, but this is required for A650 to work.

Eventually the bw table should be filled by querying the interconnect
driver for each BW in the dts, but use these dummy tables for now.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

A650 has a separate RSCC region, so dump RSCC registers separately, reading
them from the RSCC base. Without this change a GPU hang will cause a system
reset if CONFIG_DEV_COREDUMP is enabled.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

On targets where GMU is available, GMU takes over the ownership of GX GDSC
during its initialization. So, move the refcount-get on GX PD before we
initialize the GMU. This ensures that nobody can collapse the GX GDSC
once GMU owns the GX GDSC. This patch fixes some GMU OOB errors seen
during GPU wake up during a system resume.
Reported-by: NMatthias Kaehlcke <mka@chromium.org>
Signed-off-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Tested-by: NMatthias Kaehlcke <mka@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

This patch changes the plumbing to send the devfreq recommended opp rather
than the frequency. Also consolidate and rearrange the code in a6xx to set
the GPU frequency and the icc vote in preparation for the upcoming
changes for GPU->DDR scaling votes.
Signed-off-by: NSharat Masetty <smasetty@codeaurora.org>
Signed-off-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

If there is no interconnect-names, but there is an interconnects
property, then of_icc_get(dev, "gfx-mem"); would return an error
rather than NULL.

Also, if there is no interconnect-names property, there will never
be a ocmem path.  But of_icc_get(dev, "ocmem") would return -EINVAL
instead of -ENODATA.  Just don't bother trying in this case.

v2: explicity check for interconnect-names property

Fixes: 08af4769 ("drm/msm: handle for EPROBE_DEFER for of_icc_get")
Fixes: 00bb9243 ("drm/msm/gpu: add support for ocmem interconnect path")
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Check for errors instead of silently not using icc if the msm driver
probes before the interconnect driver.

Allow ENODATA for ocmem path, as it is optional and this error
is returned when "gfx-mem" path is provided but not "ocmem".

Because msm_gpu_cleanup assumes msm_gpu_init has been called, the icc path
init needs to be after msm_gpu_init for the error path to work.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>

adreno_gpu_init calls pm_runtime_enable, so adreno_gpu_cleanup needs to
call pm_runtime_disable.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>

msm_gem_address_space_create() changed to take a start/length instead
of a start/end for the iova space but all of the callers were just
cut and pasted from the old usage. Most of the mistakes have been fixed
up so just catch up the rest.

Fixes: ccac7ce3 ("drm/msm: Refactor address space initialization")
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

We don't want it under CONFIG_DRM_MSM_GPU_STATE, we need it all the
time (like the other GPUs do).

Fixes: ccac7ce3 ("drm/msm: Refactor address space initialization")
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Previously the address space went from 16M to ~0u, but with the
refactor one of the 'f's was dropped, limiting us to 256MB.
Additionally, the new interface takes a start and size, not start and
end, so we can't just copy and paste.

Fixes regressions in dEQP-VK.memory.allocation.random.*

Fixes: ccac7ce3 ("drm/msm: Refactor address space initialization")
Signed-off-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NRob Clark <robdclark@chromium.org>

This week I started seeing GPU crashes on my DragonBoard 845c
which I narrowed down to being caused by commit ccac7ce3
("drm/msm: Refactor address space initialization").

Looking through the patch, Jordan and I couldn't find anything
obviously wrong, so I ended up breaking that change up into a
number of smaller logical steps so I could figure out which part
was causing the trouble.

Ends up, visually counting 'f's is hard, esp across a number
of lines:
  0xfffffff != 0xffffffff

This patch corrects the end value we pass in to
msm_gem_address_space_create() in
adreno_iommu_create_address_space() so that it matches the value
used before the problematic patch landed.

With this change, I no longer see the GPU crashes that were
affecting me.

Cc: Shawn Guo <shawn.guo@linaro.org>
Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Jordan Crouse <jcrouse@codeaurora.org>
Cc: freedreno@lists.freedesktop.org
Fixes: ccac7ce3 ("drm/msm: Refactor address space initialization")
Signed-off-by: NJohn Stultz <john.stultz@linaro.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Also skip the newly added HFI set freq path if the GMU is powered down,
which was missing because of patches crossing paths.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Refactor how address space initialization works. Instead of having the
address space function create the MMU object (and thus require separate but
equal functions for gpummu and iommu) use a single function and pass the
MMU struct in. Make the generic code cleaner by using target specific
functions to create the address space so a2xx can do its own thing in its
own space.  For all the other targets use a generic helper to initialize
IOMMU but leave the door open for newer targets to use customization
if they need it.
Reviewed-by: NRob Clark <robdclark@gmail.com>
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Tested-by: NShawn Guo <shawn.guo@linaro.org>
[squash in rebase fixups]
Signed-off-by: NRob Clark <robdclark@chromium.org>

Everywhere an IOMMU object is created by msm_gpu_create_address_space
the IOMMU device is attached immediately after. Instead of carrying around
the infrastructure to do the attach from the device specific code do it
directly in the msm_iommu_init() function. This gets it out of the way for
more aggressive cleanups that follow.
Reviewed-by: NRob Clark <robdclark@gmail.com>
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Tested-by: NShawn Guo <shawn.guo@linaro.org>
[squash in rebase fixups and fix for unused fxn]
Signed-off-by: NRob Clark <robdclark@chromium.org>

Fixes: 8167e6fa ("drm/msm/a6xx: HFI v2 for A640 and A650")
Signed-off-by: Nkbuild test robot <lkp@intel.com>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Spelling out _unlocked for each and every driver is a annoying.
Especially if we consider how many drivers, do not know (or need to)
about the horror stories involving struct_mutex.

Just drop the suffix. It makes the API cleaner.

Done via the following script:

__from=drm_gem_object_put_unlocked
__to=drm_gem_object_put
for __file in $(git grep --name-only $__from); do
  sed -i  "s/$__from/$__to/g" $__file;
done

Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: David Airlie <airlied@linux.ie>
Signed-off-by: NEmil Velikov <emil.velikov@collabora.com>
Acked-by: NSam Ravnborg <sam@ravnborg.org>
Acked-by: NThomas Zimmermann <tzimmermann@suse.de>
Link: https://patchwork.freedesktop.org/patch/msgid/20200515095118.2743122-25-emil.l.velikov@gmail.com

Vast majority of DRM (core and drivers) are struct_mutex free.

As such we have only a handful of cases where the locked helper should
be used. Make that stand out a little bit better.

Done via the following script:

__from=drm_gem_object_put
__to=drm_gem_object_put_locked

for __file in $(git grep --name-only --word-regexp $__from); do
  sed -i  "s/\<$__from\>/$__to/g" $__file;
done

Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: linux-arm-msm@vger.kernel.org
Signed-off-by: NEmil Velikov <emil.velikov@collabora.com>
Acked-by: NSam Ravnborg <sam@ravnborg.org>
Reviewed-by: NSteven Price <steven.price@arm.com>
Acked-by: NThomas Zimmermann <tzimmermann@suse.de>
Link: https://patchwork.freedesktop.org/patch/msgid/20200515095118.2743122-12-emil.l.velikov@gmail.com

A405 device has a different set of registers than a4xx_registers.  It
has no VMIDMT or XPU registers, and VBIF registers are different.  Let's
add a405_registers for a405 device.

As adreno_is_a405() works only after adreno_gpu_init() gets called, the
assignments get moved down after adreno_gpu_init().
Signed-off-by: NShawn Guo <shawn.guo@linaro.org>
Reviewed-by: NJordan Crouse <jcrouse@codeauorora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>