Concurrent execution of the non-atomic arithmetic could result in
completely bogus values.

v2:
* Rebased on v2 of the previous patch

Cc: stable@vger.kernel.org
Bugzilla: https://bugs.freedesktop.org/106872Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of CPU invisible VRAM. Preparation for the following, no
functional change intended.

v2:
* Also change amdgpu_vram_mgr_bo_invisible_size to
  amdgpu_vram_mgr_bo_visible_size, allowing further simplification
  (Christian König)

Cc: stable@vger.kernel.org
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

so we can read/write the registers in CLK domain
through RREG32/WREG32_SOC15
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Since it seems that some vendors are storing the ATIF ACPI methods under
the same handle that ATPX lives under instead of the device's own
handle, we're going to need to be able to retrieve this handle later so
we can probe for ATIF there.
Signed-off-by: NLyude Paul <lyude@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Currently, there is nothing in amdgpu that actually uses these structs
other than amdgpu_acpi.c. Additionally, since we're about to start
saving the correct ACPI handle to use for calling ATIF in this struct
this saves us from having to handle making sure that the acpi_handle
(and by proxy, the type definition for acpi_handle and all of the other
acpi headers) doesn't need to be included within the amdgpu_drv struct
itself. This follows the example set by amdgpu_atpx_handler.c.
Signed-off-by: NLyude Paul <lyude@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Everything in the flush code path (i.e. waiting for SW queue
to become empty) names with *_flush()
and everything in the release code path names *_fini()
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Suggested-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Access to SQ_EDC_INFO requires selecting register instance and
hence mutex lock when accessing GRBM_GFX_INDEX for which a work
is schedueled from IH. But SQ interrupt can be raised on many instances
at once which means queuing work will usually succeed for the first one
but fail for the rest since the work takes time to process. To avoid
losing info about other interrupt instances call the parsing function
directly from high IRQ when current work hasn't finished and avoid
accessing SQ_EDC_INFO in that case.

v2:
Simplify high IRQ and BH handlers synchronization using work_pending.
Remove {READ,WRITE}_ONCE notations since smp_{r,w}mb are implicit
compiler barriers.

v3:
Remove exlicit memory barriers as scedule_work has r/w barriers.
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

SQ can generate interrupts on EDC/ECC errors and this struct controls
how the interrupt is handled.  The guts are filled in in the
gf_v<major>_<minor>.c files.

v2:
Rebase.
Signed-off-by: NDavid Panariti <David.Panariti@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

ISR will DRM_ERROR ECC error message.

v2:
Remove CZ only limitation.
Rebase.
Signed-off-by: NDavid Panariti <David.Panariti@amd.com>
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The ForceParWrRMW setting needs to be enabled for ECC, but disabled
when ECC is not enabled.
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enables sg display if vram size <= THRESHOLD(256M); otherwise
still use vram as display buffer.
This patch fixed some potention issues introduced by change
"allow framebuffer in GART memory as well" due to CZ/ST hardware
limitation.

v2: Change default setting to auto.
v3: Move some logic from amdgpu_display_framebuffer_domains()
    to pin function, suggested by Christian.
v4: Split into several patches.
v5: Drop module parameter for now.
Signed-off-by: NSamuel Li <Samuel.Li@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

RLC save/restore list cntl/gpm_mem/srm_mem ucodes are used for CGPG and gfxoff
function.
Signed-off-by: NHuang Rui <ray.huang@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: check reserved vram size before allocate.
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

issue:
there are VMC page fault occurred if force APP kill during
3dmark test, the cause is in entity_fini we manually signal
all those jobs in entity's queue which confuse the sync/dep
mechanism:

1)page fault occurred in sdma's clear job which operate on
shadow buffer, and shadow buffer's Gart table is cleaned by
ttm_bo_release since the fence in its reservation was fake signaled
by entity_fini() under the case of SIGKILL received.

2)page fault occurred in gfx' job because during the lifetime
of gfx job we manually fake signal all jobs from its entity
in entity_fini(), thus the unmapping/clear PTE job depend on those
result fence is satisfied and sdma start clearing the PTE and lead
to GFX page fault.

fix:
1)should at least wait all jobs already scheduled complete in entity_fini()
if SIGKILL is the case.

2)if a fence signaled and try to clear some entity's dependency, should
set this entity guilty to prevent its job really run since the dependency
is fake signaled.

v2:
splitting drm_sched_entity_fini() into two functions:
1)The first one is does the waiting, removes the entity from the
runqueue and returns an error when the process was killed.
2)The second one then goes over the entity, install it as
completion signal for the remaining jobs and signals all jobs
with an error code.

v3:
1)Replace the fini1 and fini2 with better name
2)Call the first part before the VM teardown in
amdgpu_driver_postclose_kms() and the second part
after the VM teardown
3)Keep the original function drm_sched_entity_fini to
refine the code.

v4:
1)Rename entity->finished to entity->last_scheduled;
2)Rename drm_sched_entity_fini_job_cb() to
drm_sched_entity_kill_jobs_cb();
3)Pass NULL to drm_sched_entity_fini_job_cb() if -ENOENT;
4)Replace the type of entity->fini_status with "int";
5)Remove the check about entity->finished.
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Signed-off-by: NEmily Deng <Emily.Deng@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This callback writes a value to a register and then reads
back another register and waits for a value in a single
operation.

Provide a helper function using two operations for engines
that don't support this opertion.
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Userspace needs to query this value to work around a hw bug in
certain cases.
Acked-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Allow us to determine at the soc level whether the
asic requires full reset or if soft reset will work.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König<christian.koenig@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Also remove code duplication in write and read regs functions.
This also fixes potential missing unlock in amdgpu_debugfs_regs_write
in case get_user would fail.

v2: Add SRBM mutex locking.
v3: Fix TO counter and fix comment location.
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

1. delete amdgpu_powerplay.c used for wrapping smu ip functions
2. delete struct pp_instance,
3. make struct hwmgr as the smu hw handle.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

No need to replicate it in several places.
Reviewed-by: NRex Zhu <rezhu@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Drop the "kernel" and sg parameter and give the BO type to create
explicit to amdgpu_bo_create instead of figuring it out from the
parameters.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NRoger He <Hongbo.He@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

mailbox registers can be accessed with a byte boundry according
to BIF team, so this patch prepares register byte access
and will be used by following patches.

Actually, for mailbox registers once the byte field is touched even not changed,
the mailbox behaves, so we need the byte width accessing to those sort of regs.
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NPixel Ding <Pixel.Ding@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Those belong to the TTM handling.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

1)create a routine "handle_vram_lost" to do the vram
recovery, and put it into amdgpu_device_reset/reset_sriov,
this way no need of the extra paramter to hold the
VRAM LOST information and the related macros can be removed.

3)show vram_recover failure if time out, and set TMO equal to
lockup_timeout if vram_recover is under SRIOV runtime mode.

4)report error if any ip reset failed for SR-IOV
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

fix:
should do right shift on wb before clearing

cleanups:
1,should memset all wb buffer
2,set max wb number to 128 (total 4KB) is big enough
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

fix:
should do right shift on wb before clearing

cleanups:
1,should memset all wb buffer
2,set max wb number to 128 (total 4KB) is big enough
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We have a global dummy page in TTM, use that one instead of allocating a
new one.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Instead of the pin/unpin callback implement the attach/detach ones.

Functional identical, but allows us access to the attachment.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Stuffing the PASID mapping into the VM flush isn't flexible enough since
the PASID mapping changes not as often as we need a VM flush.

v2: add missing use of gmc_v7_0_emit_pasid_mapping
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add common smu_soc_asic_init function to emulate the sillicon post sequence
Signed-off-by: NShaoyun Liu <Shaoyun.Liu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add amdgpu_emu_mode module parameter to control the emulation mode
Avoid vbios operation on emulation since there is no vbios post duirng emulation,
use the common hw_init to simulate the post
Signed-off-by: NShaoyun Liu <Shaoyun.Liu@amd.com>
Acked-By: NAlex Deucher <alexander.deucher@amd.com>
Acked-By: NChristian Konig <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

No longer used since we changed the MC programming sequence.
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Allows us to wait for a register value/mask on a ring.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <felix.kuehling@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This reverts commit 7bdc53f9 and commit
330df03b.

Neither are needed any more.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Rename as amdgpu_display_update_priority for consistency.
Signed-off-by: NSamuel Li <Samuel.Li@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>