Add a new cache mode for creating coherent host-cached BOs.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Link: https://lore.kernel.org/r/20210423190833.25319-5-jonathan@marek.caSigned-off-by: NRob Clark <robdclark@chromium.org>

msm_gem_get_vaddr() currently always maps as writecombine, so use the right
flag instead of relying on broken behavior (things don't actually work if
they are mapped as uncached).
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Acked-by: NJordan Crouse <jordan@cosmicpenguin.net>
Link: https://lore.kernel.org/r/20210423190833.25319-3-jonathan@marek.caSigned-off-by: NRob Clark <robdclark@chromium.org>

mmu500 targets don't have a "cx_mem" region, set llc_mmio to NULL in that
case to avoid the IS_ERR() condition in a6xx_llc_activate().

Fixes: 3d247123 ("drm/msm/a6xx: Add support for using system cache on MMU500 based targets")
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Link: https://lore.kernel.org/r/20210424014927.1661-1-jonathan@marek.caSigned-off-by: NRob Clark <robdclark@chromium.org>

Performance counts, and ALWAYS_ON counters used for capturing GPU
timestamps, lose their state across suspend/resume cycles.  Userspace
tooling for performance monitoring needs to be aware of this.  For
example, after a suspend userspace needs to recalibrate it's offset
between CPU and GPU time.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Acked-by: NJordan Crouse <jordan@cosmicpenguin.net>
Link: https://lore.kernel.org/r/20210325012358.1759770-3-robdclark@gmail.comSigned-off-by: NRob Clark <robdclark@chromium.org>

We were not programing the correct bit while clearing the perfcounter oob.
So, clear it correctly using the new 'clear' bit. This fixes the below
error:

[drm:a6xx_gmu_set_oob] *ERROR* Timeout waiting for GMU OOB set PERFCOUNTER: 0x80000000
Signed-off-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Link: https://lore.kernel.org/r/1617630433-36506-1-git-send-email-akhilpo@codeaurora.orgSigned-off-by: NRob Clark <robdclark@chromium.org>

Use an explicit set_pgtable_quirks method instead that just passes
the actual quirk bitmask instead.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-20-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

I suppose the microcode version check for a650 is incorrect. It checks
for the version 1.95, while the firmware released have major version of 0:
0.91 (vulnerable), 0.99 (fixing the issue).

Lower version requirements to accept firmware 0.99.

Fixes: 8490f02a ("drm/msm: a6xx: Make sure the SQE microcode is safe")
Cc: Akhil P Oommen <akhilpo@codeaurora.org>
Cc: Jordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NDmitry Baryshkov <dmitry.baryshkov@linaro.org>
Acked-by: NJordan Crouse <jordan@cosmicpenguin.net>
Message-Id: <20210331140223.3771449b-1-dmitry.baryshkov@linaro.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

They were reading a counter that was configured to ALWAYS_COUNT (ie.
cycles that the GPU is doing something) rather than ALWAYS_ON.  This
isn't the thing that userspace is looking for.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Acked-by: NJordan Crouse <jordan@cosmicpenguin.net>
Message-Id: <20210325012358.1759770-2-robdclark@gmail.com>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Commit 7bf168c8  ("drm/msm: Fix speed-bin support not to
access outside valid memory"), reworked the nvmem reading of
"speed_bin", but in doing so dropped handling of the -ENOENT
case which was previously documented as "fine".

That change resulted in the db845c board display to fail to
start, with the following error:

adreno 5000000.gpu: [drm:a6xx_gpu_init] *ERROR* failed to read speed-bin (-2). Some OPPs may not be supported by hardware

Thus, this patch simply re-adds the ENOENT handling so the lack
of the speed_bin entry isn't fatal for display, and gets things
working on db845c.

Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: Jordan Crouse <jcrouse@codeaurora.org>
Cc: Eric Anholt <eric@anholt.net>
Cc: Douglas Anderson <dianders@chromium.org>
Cc: linux-arm-msm@vger.kernel.org
Cc: freedreno@lists.freedesktop.org
Cc: Bjorn Andersson <bjorn.andersson@linaro.org>
Cc: YongQin Liu <yongqin.liu@linaro.org>
Reported-by: NYongQin Liu <yongqin.liu@linaro.org>
Fixes: 7bf168c8  ("drm/msm: Fix speed-bin support not to access outside valid memory")
Signed-off-by: NJohn Stultz <john.stultz@linaro.org>
Reviewed-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Reviewed-by: NDouglas Anderson <dianders@chromium.org>
Message-Id: <20210330013408.2532048-1-john.stultz@linaro.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

While passing the A530-specific lm_setup func to A530 and A540
to !A530 was fine back when only these two were supported, it
certainly is not a good idea to send A540 specifics to smaller
GPUs like A508 and friends.
Signed-off-by: NKonrad Dybcio <konrad.dybcio@somainline.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

When running the latest kernel on an sc7180 with KASAN I got this
splat:
  BUG: KASAN: slab-out-of-bounds in a6xx_gpu_init+0x618/0x644
  Read of size 4 at addr ffffff8088f36100 by task kworker/7:1/58
  CPU: 7 PID: 58 Comm: kworker/7:1 Not tainted 5.11.0+ #3
  Hardware name: Google Lazor (rev1 - 2) with LTE (DT)
  Workqueue: events deferred_probe_work_func
  Call trace:
   dump_backtrace+0x0/0x3a8
   show_stack+0x24/0x30
   dump_stack+0x174/0x1e0
   print_address_description+0x70/0x2e4
   kasan_report+0x178/0x1bc
   __asan_report_load4_noabort+0x44/0x50
   a6xx_gpu_init+0x618/0x644
   adreno_bind+0x26c/0x438

This is because the speed bin is defined like this:
  gpu_speed_bin: gpu_speed_bin@1d2 {
    reg = <0x1d2 0x2>;
    bits = <5 8>;
  };

As you can see the "length" is 2 bytes. That means that the nvmem
subsystem allocates only 2 bytes. The GPU code, however, was casting
the pointer allocated by nvmem to a (u32 *) and dereferencing. That's
not so good.

Let's fix this to just use the nvmem_cell_read_u16() accessor function
which simplifies things and also gets rid of the splat.

Let's also put an explicit conversion from little endian in place just
to make things clear. The nvmem subsystem today is assuming little
endian and this makes it clear. Specifically, the way the above sc7180
cell is interpreted:

NVMEM:
 +--------+--------+--------+--------+--------+
 | ...... | 0x1d3  | 0x1d2  | ...... | 0x000  |
 +--------+--------+--------+--------+--------+
              ^       ^
             msb     lsb

You can see that the least significant data is at the lower address
which is little endian.

NOTE: someone who is truly paying attention might wonder about me
picking the "u16" version of this accessor instead of the "u8" (since
the value is 8 bits big) or the u32 version (just for fun). At the
moment you need to pick the accessor that exactly matches the length
the cell was specified as in the device tree. Hopefully future
patches to the nvmem subsystem will fix this.

Fixes: fe7952c6 ("drm/msm: Add speed-bin support to a618 gpu")
Signed-off-by: NDouglas Anderson <dianders@chromium.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Most a6xx targets have security issues that were fixed with new versions
of the microcode(s). Make sure that we are booting with a safe version of
the microcode for the target and print a message and error if not.

v2: Add more informative error messages and fix typos
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The cleanup patch broke a6xx_gmu_clear_oob, fix it by adding the missing
bitshift operation.

Fixes: 555c50a4 ("drm/msm: Clean up GMU OOB set/clear handling")
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NEric Anholt <eric@anholt.net>
Signed-off-by: NRob Clark <robdclark@chromium.org>

dev_pm_opp_set_bw() is getting removed and dev_pm_opp_set_opp() should
be used instead. Migrate to the new API.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>

Now that the bug is fixed in the minimal way for stable, go make the
code table-driven.
Signed-off-by: NEric Anholt <eric@anholt.net>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Now that we're not racing with GPU setup, also fix races of timestamps
against other timestamps.  In freedreno CI, we were seeing this path trigger
timeouts on setting the GMU bit, producing:

[drm:_a6xx_gmu_set_oob] *ERROR* Timeout waiting for GMU OOB set GPU_SET: 0x0

and this triggered especially on the first set of tests right after
boot (it's probably easier to lose the race than one might think,
given that we start many tests in parallel, and waiting for NFS to
page in code probably means that lots of tests hit the same point of
screen init at the same time).  As of this patch, the message seems to
have completely gone away.
Signed-off-by: NEric Anholt <eric@anholt.net>
Fixes: 4b565ca5 ("drm/msm: Add A6XX device support")
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

We were using the same force-poweron bit in the two codepaths, so they
could race to have one of them lose GPU power early.

freedreno CI was seeing intermittent errors like:
[drm:_a6xx_gmu_set_oob] *ERROR* Timeout waiting for GMU OOB set GPU_SET: 0x0
and this issue could have contributed to it.
Signed-off-by: NEric Anholt <eric@anholt.net>
Fixes: 4b565ca5 ("drm/msm: Add A6XX device support")
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Port over the command from downstream to prevent undefined
behaviour.
Signed-off-by: NKonrad Dybcio <konrad.dybcio@somainline.org>
Signed-off-by: NAngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Port over the command from downstream to prevent undefined
behaviour.
Signed-off-by: NKonrad Dybcio <konrad.dybcio@somainline.org>
Signed-off-by: NAngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The upstream API for some reason uses logbase2 instead of
just passing the argument as-is, whereas downstream CAF
kernel does the latter.

Hence, a mistake has been made when porting:
4 is the value that's supposed to be passed, but
log2(4) = 2. Changing the value to 16 (= 2^4) fixes
the issue.
Signed-off-by: NKonrad Dybcio <konrad.dybcio@somainline.org>
Signed-off-by: NAngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Resetting the VBIF before power collapse is done to avoid getting
bogus FIFO entries during the suspend sequence or subsequent resume,
but this is doable only on Adreno 510 and Adreno 530, as the other
units will tendentially lock up.
Especially on Adreno 508, the GPU will show lockups and very bad
slownesses after processing the first frame.

Avoiding to execute the RBBM SW Reset before suspend will stop the
lockup issue from happening on at least Adreno 508/509/512.
Signed-off-by: NAngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The Adreno 508/509/512 GPUs are stripped versions of the Adreno
5xx found in the mid-end SoCs such as SDM630, SDM636, SDM660 and
SDA variants; these SoCs are usually provided with ZAP firmwares,
but they have no available GPMU.
Signed-off-by: NAngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
Tested-by: NMartin Botka <martin.botka1@gmail.com>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The "main" if branch where we program the other registers for the
Adreno 5xx family of GPUs should not contain the PC_DBG_ECO_CNTL
register programming because this has logical similarity
differences from all the others.

A later commit will show the entire sense of this.
Signed-off-by: NAngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The PC_DBG_ECO_CNTL register on the Adreno A5xx family gets
programmed to some different values on a per-model basis.
At least, this is what we intend to do here;

Unfortunately, though, this register is being overwritten with a
static magic number, right after applying the GPU-specific
configuration (including the GPU-specific quirks) and that is
effectively nullifying the efforts.

Let's remove the redundant and wrong write to the PC_DBG_ECO_CNTL
register in order to retain the wanted configuration for the
target GPU.
Signed-off-by: NAngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

A6XX GPUs have support for last level cache(LLC) also known
as system cache and need to set the bus attributes to
use it. Currently we use a generic adreno iommu address space
implementation which are also used by older GPU generations
which do not have LLC and might introduce issues accidentally
and is not clean in a way that anymore additions of GPUs
supporting LLC would have to be guarded under ifdefs. So keep
the generic code separate and make the address space creation
A6XX specific. We also have a helper to set the llc attributes
so that if the newer GPU generations do support them, we can
use it instead of open coding domain attribute setting for each
GPU.
Signed-off-by: NSai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Domain attribute setting for LLCC is guarded by !IS_ERR
check which works fine only when CONFIG_QCOM_LLCC=y but
when it is disabled, the LLCC apis return NULL and that
is not handled by IS_ERR check. Due to this, domain attribute
for LLCC will be set even on GPUs which do not support it
and cause issues, so correct this by using IS_ERR_OR_NULL
checks appropriately. Meanwhile also cleanup comment block
and remove unwanted blank line.

Fixes: 00fd44a1 ("drm/msm: Only enable A6xx LLCC code on A6xx")
Fixes: 474dadb8 ("drm/msm/a6xx: Add support for using system cache(LLC)")
Signed-off-by: NSai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

On at least MSM8998 it's possible to find Adreno 540.0 and 540.1
but I have never found any 540.2. In any case, the patchids 0-1
for A540 are completely supported by this driver and there is no
reason to disallow probing them (as they also share the same
firmware names).

Besides that, the patchid number is also used in the a5xx_power.c
function a540_lm_setup to disable the battery current limiter,
which makes faking the Adreno patchid to .2 (which would anyway
be sad) useless and even producing breakages.
Signed-off-by: NAngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Some GPUs support different max frequencies depending on the platform.
To identify the correct variant, we should check the gpu speedbin
fuse value. Add support for this speedbin detection to a6xx family
along with the required fuse details for a618 gpu.
Signed-off-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Using this code on A5xx (and probably older too) causes a
smmu bug.

Fixes: 474dadb8 ("drm/msm/a6xx: Add support for using system cache(LLC)")
Signed-off-by: NKonrad Dybcio <konrad.dybcio@somainline.org>
Tested-by: NAngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NSai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Using the GPU with a VRAM Carveout is a security vulnerability.
Nevertheless it is sometimes required, especially when no IOMMU
implementation is available for a certain platform.
Signed-off-by: NIskren Chernev <iskren.chernev@gmail.com>
Signed-off-by: NRob Clark <robdclark@chromium.org>

nr_rings is reset to 1, but when this function is called for a second
(and third!) time nr_rings > 1 is false, thus the else case is entered
to set up a buffer for the RPTR shadow and consequently written to
RB_RPTR_ADDR, hanging platforms without WHERE_AM_I firmware support.

Restructure the condition in such a way that shadow buffer setup only
ever happens when has_whereami is true; otherwise preemption is only
finalized when the number of ring buffers has not been reset to 1 yet.

Fixes: 8907afb4 ("drm/msm: Allow a5xx to mark the RPTR shadow as privileged")
Signed-off-by: NMarijn Suijten <marijn.suijten@somainline.org>
Tested-by: NAngeloGioacchino Del Regno <angelogioacchino.delregno@somainline.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

GPU targets with an MMU-500 attached have a slightly different process for
enabling system cache. Use the compatible string on the IOMMU phandle
to see if an MMU-500 is attached and modify the programming sequence
accordingly.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NSai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The last level system cache can be partitioned to 32 different
slices of which GPU has two slices preallocated. One slice is
used for caching GPU buffers and the other slice is used for
caching the GPU SMMU pagetables. This talks to the core system
cache driver to acquire the slice handles, configure the SCID's
to those slices and activates and deactivates the slices upon
GPU power collapse and restore.

Some support from the IOMMU driver is also needed to make use
of the system cache to set the right TCR attributes. GPU then
has the ability to override a few cacheability parameters which
it does to override write-allocate to write-no-allocate as the
GPU hardware does not benefit much from it.

DOMAIN_ATTR_IO_PGTABLE_CFG is another domain level attribute used
by the IOMMU driver for pagetable configuration which will be used
to set a quirk initially to set the right attributes to cache the
hardware pagetables into the system cache.
Signed-off-by: NSharat Masetty <smasetty@codeaurora.org>
[saiprakash.ranjan: fix to set attr before device attach to iommu and rebase]
Signed-off-by: NSai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Fixes the following W=1 kernel build warning(s):

 drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c:83:7: warning: no previous prototype for ‘state_kcalloc’ [-Wmissing-prototypes]
 drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c:95:7: warning: no previous prototype for ‘state_kmemdup’ [-Wmissing-prototypes]
 drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c:947:6: warning: no previous prototype for ‘a6xx_gpu_state_destroy’ [-Wmissing-prototypes]

Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: linux-arm-msm@vger.kernel.org
Cc: dri-devel@lists.freedesktop.org
Cc: freedreno@lists.freedesktop.org
Signed-off-by: NLee Jones <lee.jones@linaro.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Fixes the following W=1 kernel build warning(s):

 drivers/gpu/drm/msm/adreno/a6xx_gpu.c:33:6: warning: no previous prototype for ‘a6xx_idle’ [-Wmissing-prototypes]

Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: linux-arm-msm@vger.kernel.org
Cc: dri-devel@lists.freedesktop.org
Cc: freedreno@lists.freedesktop.org
Signed-off-by: NLee Jones <lee.jones@linaro.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Similar to the previous patch, clear shadow on suspend to avoid timeouts
waiting for ringbuffer space.

Fixes: 8907afb4 ("drm/msm: Allow a5xx to mark the RPTR shadow as privileged")
Signed-off-by: NRob Clark <robdclark@chromium.org>

Clear the shadow rptr on suspend.  Otherwise, when we resume, we can
have a stale value until CP_WHERE_AM_I executes.  If we suspend near
the ringbuffer wraparound point, this can lead to a chicken/egg
situation where we are waiting for ringbuffer space to write the
CP_WHERE_AM_I (or CP_INIT) packet, because we mistakenly believe that
the ringbuffer is full (due to stale rptr value in the shadow).

Fixes errors like:

  [drm:adreno_wait_ring [msm]] *ERROR* timeout waiting for space in ringbuffer 0

in the resume path.

Fixes: d3a569fc ("drm/msm: a6xx: Use WHERE_AM_I for eligible targets")
Signed-off-by: NRob Clark <robdclark@chromium.org>

Before adding another lock, give ring->lock a more descriptive name.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NKristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The microcode bo's should never be madvise(WONTNEED), so these should
not be using msm_gem_get_vaddr_active().
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NKristian H. Kristensen <hoegsberg@google.com>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The dev_pm_opp_of_add_table() api initializes the icc nodes for gpu
indirectly. So we can avoid using of_icc_get() api in the common
probe path. To improve this, move of_icc_get() to target specific code
where it is required.

This patch helps to fix duplicate gpu node listed in the interconnect
summary from the debugfs.
Signed-off-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>