[ Upstream commit 3bfa8897e4d08f822d1d58cf6cbbffbccef82e08 ]

Instead of delaying that to the first query. Otherwise we could try to use the
SDMA for VM updates before the IB tests are done.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NSasha Levin <sashal@kernel.org>

The RCU grace period is harmless and avoiding it is not worth the effort
of doubling the implementation.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming  Zhou <david1.zhou@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Properly handle cases where one or more instance of the IP
block may be harvested.

v2: make sure ip_num_rings is initialized amdgpu_queue_mgr.c
v3: rebase on Christian's UVD changes, drop unused var
Reviewed-by: NJames Zhu <James.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Going to completely rework the context to ring mapping with Nayan's GSoC
work, but for now just stopping to expose the second UVD instance should
do it.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

No need to bitcast a boolean and even if we should use "!!" instead.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Concurrent execution of the non-atomic arithmetic could result in
completely bogus values.

v2:
* Rebased on v2 of the previous patch

Cc: stable@vger.kernel.org
Bugzilla: https://bugs.freedesktop.org/106872Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of CPU invisible VRAM. Preparation for the following, no
functional change intended.

v2:
* Also change amdgpu_vram_mgr_bo_invisible_size to
  amdgpu_vram_mgr_bo_visible_size, allowing further simplification
  (Christian König)

Cc: stable@vger.kernel.org
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

PAGE_SIZE for start_alignment is far much than hw requirement,
And now, update to expereince value from window side.
Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NMarek Olšák <marek.olsak@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Acked-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add AMDGPU_HW_IP_VCN_JPEG to info query
Signed-off-by: NBoyuan Zhang <boyuan.zhang@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

With this we can now terminate jobs enqueue into SW queue the moment
the task is being killed instead of waiting for last user of
drm file to release it.

Also stop checking for kref_read(&ctx->refcount) == 1 when
calling drm_sched_entity_do_release since other task
might still hold a reference to this entity but we don't
care since KILL means terminate job submission regardless
of what other tasks are doing.

v2:
Use returned remaining timeout as parameter for the next call.
Rebase.

v3:
Switch to working with jiffies.
Streamline remainder TO usage.
Rebase.

v4:
Rebase.
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Vega20 has dual-UVD. Need add multiple instances support for uvd.
Restruct uvd.inst, using uvd.inst[0] to replace uvd.inst->.
Repurpose amdgpu_ring::me for instance index, and initialize to 0.
There are no any logical changes here.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Vega20 has dual-UVD. Need Restruct amdgpu_device::uvd to support
multiple uvds. There are no any logical changes here.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

RLC save/restore list cntl/gpm_mem/srm_mem ucodes are used for CGPG and gfxoff
function.
Signed-off-by: NHuang Rui <ray.huang@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Useful for info gathering about what firmwares are in use in
the driver.
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

amdgpu_ib_ring_tests() runs test IB's on rings at boot
contributes to ~500 ms of amdgpu driver's boot time.

This patch defers it and ensures that its executed
in amdgpu_info_ioctl() if it wasn't scheduled.

V2: Use queue_delayed_work() & flush_delayed_work().
V3: removed usage of separate wq, ensure ib tests is
    run before enabling clockgating.
Signed-off-by: NShirish S <shirish.s@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Currently smu only calculate average gpu power in real time.

for vddc/vddci/max power,
User need to set start time and end time, firmware can calculate
the average vddc/vddci/max power. but the type of return values
is not unified. For Vi, return type is uint.
For vega, return type is float.

so this struct can't be suitable for all asics.
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

issue:
there are VMC page fault occurred if force APP kill during
3dmark test, the cause is in entity_fini we manually signal
all those jobs in entity's queue which confuse the sync/dep
mechanism:

1)page fault occurred in sdma's clear job which operate on
shadow buffer, and shadow buffer's Gart table is cleaned by
ttm_bo_release since the fence in its reservation was fake signaled
by entity_fini() under the case of SIGKILL received.

2)page fault occurred in gfx' job because during the lifetime
of gfx job we manually fake signal all jobs from its entity
in entity_fini(), thus the unmapping/clear PTE job depend on those
result fence is satisfied and sdma start clearing the PTE and lead
to GFX page fault.

fix:
1)should at least wait all jobs already scheduled complete in entity_fini()
if SIGKILL is the case.

2)if a fence signaled and try to clear some entity's dependency, should
set this entity guilty to prevent its job really run since the dependency
is fake signaled.

v2:
splitting drm_sched_entity_fini() into two functions:
1)The first one is does the waiting, removes the entity from the
runqueue and returns an error when the process was killed.
2)The second one then goes over the entity, install it as
completion signal for the remaining jobs and signals all jobs
with an error code.

v3:
1)Replace the fini1 and fini2 with better name
2)Call the first part before the VM teardown in
amdgpu_driver_postclose_kms() and the second part
after the VM teardown
3)Keep the original function drm_sched_entity_fini to
refine the code.

v4:
1)Rename entity->finished to entity->last_scheduled;
2)Rename drm_sched_entity_fini_job_cb() to
drm_sched_entity_kill_jobs_cb();
3)Pass NULL to drm_sched_entity_fini_job_cb() if -ENOENT;
4)Replace the type of entity->fini_status with "int";
5)Remove the check about entity->finished.
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Signed-off-by: NEmily Deng <Emily.Deng@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

driver will set dpm_enabled to true only when
module parameter amdgpu_dpm not equal to 0 and
smu hw initialize successfully.
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Need to be able to query the VCN firmware version from
userspace to determine supported features, etc.
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Acked-by: NLeo Liu <leo.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Older VCE FW versions are buggy and can't work with 48bit address
spaces.

RFC: Should we limit the address space or just reject loading the older
VCE firmware?
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Reserve VA space at the top for older generations as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Acked-by: NMonk Liu <monk.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add display to the name for consistency.
Signed-off-by: NSamuel Li <Samuel.Li@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add display to the name for consistency.
Signed-off-by: NSamuel Li <Samuel.Li@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add sub-queries for stable pstate shader/memory clock.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Start to always allocate a pasid for each VM.

v2: use dev_warn when we run out of PASIDs
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

And rename it to amdgpu_gmc as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NSamuel Li <Samuel.Li@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This driver can use drm_fb_helper_lastclose() in its .lastclose function.
It can also use drm_fb_helper_output_poll_changed() as its
.output_poll_changed callback.
Remove the unused driver implementations.

Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Signed-off-by: NNoralf Trønnes <noralf@tronnes.org>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Let userspace know how much area we have above the 48bit VA hole on
Vega10.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Similar to the CPU address space the VA on Vega10 has a hole in it.

v2: use dev_dbg instead of dev_err
v3: add some more comments to explain how the hw works
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
CC: stable@vger.kernel.org
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Retry at drm_dev_register instead of amdgpu_device_init.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NPixel Ding <Pixel.Ding@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: Npding <Pixel.Ding@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Also finalize kfd inside amdgpu_device_fini. kfd device_init needs
SRIOV exclusive accessing. Try to gather exclusive accessing to
reduce time consuming.
Signed-off-by: Npding <Pixel.Ding@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The exclusive mode has real-time limitation in reality, such like being
done in 300ms. It's easy observed if running many VF/VMs in single host
with heavy CPU workload.

If we find the init fails due to exclusive mode timeout, try it again.

v2:
 - rewrite the condition for readable value.

v3:
 - fix typo, add comments for sleep
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: Npding <Pixel.Ding@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Allows userspace to figure out if VRAM was lost.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of per device track the VRAM lost per context and return ECANCELED
instead of ENODEV.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Keep blocking the CS, but revert everything else. Mapping BOs and info IOCTL
are harmless and can still happen even when VRAM content ist lost.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The AMDGPU_SCHED_OP_PROCESS_PRIORITY_OVERRIDE ioctls are used to set
the priority of a different process in the current system.

When a request is dropped, the process's contexts will be
restored to the priority specified at context creation time.

A request can be dropped by setting the override priority to
AMDGPU_CTX_PRIORITY_UNSET.

An fd is used to identify the remote process. This is simpler than
passing a pid number, which is vulnerable to re-use, etc.

This functionality is limited to DRM_MASTER since abuse of this
interface can have a negative impact on the system's performance.

v2: removed unused output structure
v3: change refcounted interface for a regular set operation
Signed-off-by: NAndres Rodriguez <andresx7@gmail.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

for being able to convert an amdgpu fence into one of the handles.
Mesa will use this.
Reviewed-by: NDave Airlie <airlied@redhat.com>
Signed-off-by: NMarek Olšák <marek.olsak@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Supported DCE versions: 8.0, 10.0, 11.0, 11.2

v2: rebase against 4.11
Signed-off-by: NHarry Wentland <harry.wentland@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>