In $debugfs/gem we already show any vma(s) associated with an object.
Also show process names if the vma's address space is a per-process
address space.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>

Add support for allocating private address space instances. Targets that
support per-context pagetables should implement their own function to
allocate private address spaces.

The default will return a pointer to the global address space.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>

Now that we can get the ctx from the submitqueue, the extra arg is
redundant.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
[split out of previous patch to reduce churny noise]
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>

This will be populated by adreno-smmu, to provide a way for coordinating
enabling/disabling TTBR0 translation.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>

In a later patch, the drvdata will not directly be 'struct msm_gpu *',
so add a helper to reduce the churn.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>

Signed-off-by: NRob Clark <robdclark@chromium.org>

Technically the GMU specific one is a bit redundant, but it was useful
to track down a bug.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>

a650 supports expanded apriv support that allows us to map critical buffers
(ringbuffer and memstore) as as privileged to protect them from corruption.

Cc: stable@vger.kernel.org
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

This patch changes the plumbing to send the devfreq recommended opp rather
than the frequency. Also consolidate and rearrange the code in a6xx to set
the GPU frequency and the icc vote in preparation for the upcoming
changes for GPU->DDR scaling votes.
Signed-off-by: NSharat Masetty <smasetty@codeaurora.org>
Signed-off-by: NAkhil P Oommen <akhilpo@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

These never get set back to 0 when probing fails, so an attempt to probe
again results in broken behavior. Fix the problem by setting thse to zero
before they are used.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Refactor how address space initialization works. Instead of having the
address space function create the MMU object (and thus require separate but
equal functions for gpummu and iommu) use a single function and pass the
MMU struct in. Make the generic code cleaner by using target specific
functions to create the address space so a2xx can do its own thing in its
own space.  For all the other targets use a generic helper to initialize
IOMMU but leave the door open for newer targets to use customization
if they need it.
Reviewed-by: NRob Clark <robdclark@gmail.com>
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Tested-by: NShawn Guo <shawn.guo@linaro.org>
[squash in rebase fixups]
Signed-off-by: NRob Clark <robdclark@chromium.org>

Everywhere an IOMMU object is created by msm_gpu_create_address_space
the IOMMU device is attached immediately after. Instead of carrying around
the infrastructure to do the attach from the device specific code do it
directly in the msm_iommu_init() function. This gets it out of the way for
more aggressive cleanups that follow.
Reviewed-by: NRob Clark <robdclark@gmail.com>
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Tested-by: NShawn Guo <shawn.guo@linaro.org>
[squash in rebase fixups and fix for unused fxn]
Signed-off-by: NRob Clark <robdclark@chromium.org>

Vast majority of DRM (core and drivers) are struct_mutex free.

As such we have only a handful of cases where the locked helper should
be used. Make that stand out a little bit better.

Done via the following script:

__from=drm_gem_object_put
__to=drm_gem_object_put_locked

for __file in $(git grep --name-only --word-regexp $__from); do
  sed -i  "s/\<$__from\>/$__to/g" $__file;
done

Cc: Rob Clark <robdclark@gmail.com>
Cc: Sean Paul <sean@poorly.run>
Cc: linux-arm-msm@vger.kernel.org
Signed-off-by: NEmil Velikov <emil.velikov@collabora.com>
Acked-by: NSam Ravnborg <sam@ravnborg.org>
Reviewed-by: NSteven Price <steven.price@arm.com>
Acked-by: NThomas Zimmermann <tzimmermann@suse.de>
Link: https://patchwork.freedesktop.org/patch/msgid/20200515095118.2743122-12-emil.l.velikov@gmail.com

Also log buffers with the DUMP flag set, to ensure we capture all useful
cmdstream in crashdump state with modern mesa.

Otherwise we miss out on the contents of "state object" cmdstream
buffers.

v2: add missing 'inline'
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>

Without this header file, compile-testing may run into a missing
declaration:

drivers/gpu/drm/msm/msm_gpu.c:444:4: error: implicit declaration of function 'put_task_struct' [-Werror,-Wimplicit-function-declaration]

Fixes: 482f9632 ("drm/msm: Fix task dump in gpu recovery")
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

The arguments related to IOMMU port name have been unused since
commit 944fc36c ("drm/msm: use upstream iommu") and can be removed.
Signed-off-by: NDrew Davenport <ddavenport@chromium.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Remove the homebrewed bulk clock get function and replace it with
devm_clk_bulk_get_all().
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NStephen Boyd <swboyd@chromium.org>
Reviewed-by: NRob Clark <robdclark@gmail.com>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Since governor name is defined by DEVFREQ framework internally, use the
macro definition instead of using the name directly.
Signed-off-by: NYue Hu <huyue2@yulong.com>
Reviewed-by: NChanwoo Choi <cw00.choi@samsung.com>
Acked-by: Jordan Crouse <jcrouse@codeaurora.org> for the msm part.
Signed-off-by: NRob Herring <robh@kernel.org>
Link: https://patchwork.freedesktop.org/patch/msgid/20190725035239.1192-1-zbestahu@gmail.com

Based on 1 normalized pattern(s):

  this program is free software you can redistribute it and or modify
  it under the terms of the gnu general public license version 2 as
  published by the free software foundation this program is
  distributed in the hope that it will be useful but without any
  warranty without even the implied warranty of merchantability or
  fitness for a particular purpose see the gnu general public license
  for more details you should have received a copy of the gnu general
  public license along with this program if not see http www gnu org
  licenses

extracted by the scancode license scanner the SPDX license identifier

  GPL-2.0-only

has been chosen to replace the boilerplate/reference in 503 file(s).
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Reviewed-by: NAlexios Zavras <alexios.zavras@intel.com>
Reviewed-by: NAllison Randal <allison@lohutok.net>
Reviewed-by: NEnrico Weigelt <info@metux.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190602204653.811534538@linutronix.deSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Pass the index of the MMU domain in struct msm_file_private instead
of assuming gpu->id throughout the submit path. This clears the way
to change ctx->aspace to a per-instance pagetable.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Now that we don't have the mmap_sem lock inversion, we don't need to
jump through this particular hoop anymore.
Signed-off-by: NKristian H. Kristensen <hoegsberg@chromium.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

Add the capability to query information from a submit queue.
The first available parameter is for querying the number of GPU faults
(hangs) that can be attributed to the queue.

This is useful for implementing context robustness. A user context can
regularly query the number of faults to see if it is responsible for any
and if so it can invalidate itself.

This is also helpful for testing by confirming to the user  driver if a
particular command stream caused a fault (or not as the case may be).
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@chromium.org>

For KHR_robustness, userspace wants to know two things, the count of GPU
faults globally, and the count of faults attributed to a given context.
This patch providees the former, and the next patch provides the latter.
Signed-off-by: NRob Clark <robdclark@chromium.org>
Reviewed-by: NJordan Crouse <jcrouse@codeaurora.org>

Every GPU core only has one interrupt so there isn't any
value in looking up the interrupt by name. Remove the name (which
is legacy anyway) and use platform_get_irq() instead.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NDouglas Anderson <dianders@chromium.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>

Every GPU core only has one interrupt so there isn't any
value in looking up the interrupt by name. Remove the name (which
is legacy anyway) and use platform_get_irq() instead.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NDouglas Anderson <dianders@chromium.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>

A2XX has its own very simple MMU.

Added a msm_use_mmu() function because we can't rely on iommu_present to
decide to use MMU or not.
Signed-off-by: NJonathan Marek <jonathan@marek.ca>
Signed-off-by: NRob Clark <robdclark@gmail.com>

For debugging purposes it is useful to assign descriptions
to buffers so that we know what they are used for. Add
a field to the buffer object and use that to name the various
kernel side allocations which ends up looking like like this
in /d/dri/X/gem:

   flags       id ref  offset   kaddr            size     madv      name
   00040000: I  0 ( 1) 00000000 0000000070b79eca 00004096           memptrs
      vmas: [gpu: 01000000,mapped,inuse=1]
   00020000: I  0 ( 1) 00000000 0000000031ed4074 00032768           ring0
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>

Add a reference count to track how many times a particular
chunk of iova memory is pinned (mapped) in the iomu and
add msm_gem_unpin_iova to give up references.

It is important to note that msm_gem_unpin_iova replaces
msm_gem_put_iova because the new implicit behavior
that an assigned iova in a given vma is now valid for the
life of the buffer and what we are really focusing on is
the use of that iova.

For now the unmappings are lazy; once the reference counts
go to zero they *COULD* be unmapped dynamically but that
will require an outside force such as a shrinker or
mm_notifiers.  For now, we're just focusing on getting
the counting right and setting ourselves up to be ready
for the future.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>

Add a new function to get and pin the iova memory in one
step (basically renaming the old msm_gem_get_iova function)
and switch msm_gem_get_iova() to only allocate an iova but
not map it in the IOMMU. This is only currently used by
msm_ioctl_gem_info() since all other users of of the iova
expect that the memory be immediately available.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>

Buffer objects allocated with msm_gem_kernel_new() are mostly
freed the same way so we can save a few lines of code with a
common function.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>

Instead of trying to store all the tagged buffers from a hanging
submit only store the command buffers that were not imported.
This cuts down on the amount of data stored in the GPU state to
the base minimum of useful information.

The downside is that this will make it more difficult to
successfully replay a hang with just the GPU state but there
isn't any reason why that functionality can't be added back
in later once we've figured out how to better communicate
such massive amounts of data.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>

Add trace events to track the progress of a GPU submission
msm_gpu_submit occurs at the beginning of the submissions,
msm_gpu_submit_flush happens when the submission is put on
the ringbuffer and msm_submit_flush_retired is sent when
the operation is retired.

To make it easier to track the operations a unique sequence
number is assigned to each submission and displayed in each
event output so a human or a script can easily associate
the events related to a specific submission.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>

Allocate the correct buffer size for the GPU memptrs. The incorrect
size hasn't affected us thus far since the incorrect size was larger
than the intended size and we're still stuck on page sized
granularity anyway but technically correct is the best kind of
correct.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>

Use DRM_DEV_INFO/ERROR/WARN instead of dev_info/err/debug to generate
drm-formatted specific log messages so that it will be easy to
differentiate in case of multiple instances of driver.
Signed-off-by: NMamta Shukla <mamtashukla555@gmail.com>
Signed-off-by: NRob Clark <robdclark@gmail.com>

The current recovery code gets a pointer to the task struct and does a
few things all within the rcu_read_lock. This puts constraints on the
types of gfp flags that can be used within the rcu lock. This patch
instead gets a reference to the task within the rcu lock and releases
the lock immediately, this way the task stays afloat until we need it and
we also get to use the desired gfp flags.
Signed-off-by: NSharat Masetty <smasetty@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>
Signed-off-by: NSean Paul <seanpaul@chromium.org>

This patch simply checks first to see if the target can support crash dump
capture before proceeding.
Signed-off-by: NSharat Masetty <smasetty@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>
Signed-off-by: NSean Paul <seanpaul@chromium.org>

The devfreq framework requires the drivers to provide busy time estimations.
The GPU driver relies on the hardware performance counteres for the busy time
estimations, but different hardware revisions have counters which can be
sourced from different clocks. So the busy time estimation will be target
dependent.  Additionally on targets where the clocks are completely controlled
by the on chip microcontroller, fetching and setting the current GPU frequency
will be different. This patch aims to embrace these differences by re-factoring
the devfreq code a bit.
Signed-off-by: NSharat Masetty <smasetty@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>

Devfreq turns on and starts recommending power level as soon as it is
initialized. The GPU is still not powered on by the time the devfreq
init happens and this leads to problems on GPU's where register access
is needed to get/set power levels. So we start suspended and only restart
devfreq when GPU is powered on.
Signed-off-by: NSharat Masetty <smasetty@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>

When CONFIG_DEV_COREDUMP isn't defined msm_gpu_crashstate_capture
doesn't pass the correct parameters.
drivers/gpu/drm/msm/msm_gpu.c: In function ‘recover_worker’:
drivers/gpu/drm/msm/msm_gpu.c:479:34: error: passing argument 2 of ‘msm_gpu_crashstate_capture’ from incompatible pointer type [-Werror=incompatible-pointer-types]
  msm_gpu_crashstate_capture(gpu, submit, comm, cmd);
                                  ^~~~~~
drivers/gpu/drm/msm/msm_gpu.c:388:13: note: expected ‘char *’ but argument is of type ‘struct msm_gem_submit *’
 static void msm_gpu_crashstate_capture(struct msm_gpu *gpu, char *comm,
             ^~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/gpu/drm/msm/msm_gpu.c:479:2: error: too many arguments to function ‘msm_gpu_crashstate_capture’
  msm_gpu_crashstate_capture(gpu, submit, comm, cmd);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/gpu/drm/msm/msm_gpu.c:388:13: note: declared here
 static void msm_gpu_crashstate_capture(struct msm_gpu *gpu, char *comm,

In current code the function msm_gpu_crashstate_capture parameters.

Fixes: cdb95931 ("drm/msm/gpu: Add the buffer objects from the submit to the crash dump")
Signed-off-by: NAnders Roxell <anders.roxell@linaro.org>
Reviewed-By: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>

Add support for the A6XX family of Adreno GPUs. The biggest addition
is the GMU (Graphics Management Unit) which takes over most of the
power management of the GPU itself but in a ironic twist of fate
needs a goodly amount of management itself. Add support for the
A6XX core code, the GMU and the HFI (hardware firmware interface)
queue that the CPU uses to communicate with the GMU.
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Signed-off-by: NRob Clark <robdclark@gmail.com>