The SOC15_REG_OFFSET() macro wasn't used, making the soft recovery fail.

v2: use WREG32_SOC15 instead of WREG32 + SOC15_REG_OFFSET
Signed-off-by: NPierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

The GDS and GWS blocks default to allowing all VMIDs to
access all entries. Graphics VMIDs can handle setting
these limits when the driver launches work. However,
compute workloads under HWS control don't go through the
kernel driver. Instead, HWS firmware should set these
limits when a process is put into a VMID slot.

Disable access to these devices by default by turning off
all mask bits (for OA) and setting BASE=SIZE=0 (for GDS
and GWS) for all compute VMIDs. If a process wants to use
these resources, they can request this from the HWS
firmware (when such capabilities are enabled). HWS will
then handle setting the base and limit for the process when
it is assigned to a VMID.

This will also prevent user kernels from getting 'stuck' in
GWS by accident if they write GWS-using code but HWS
firmware is not set up to handle GWS reset. Until HWS is
enabled to handle GWS properly, all GWS accesses will
MEM_VIOL fault the kernel.

v2: Move initialization outside of SRBM mutex
Signed-off-by: NJoseph Greathouse <Joseph.Greathouse@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add 5 bits to the offset for SRBM selection to handle VMIDs.  Also
update the select_me_pipe_q() callback to also select VMID.
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Apply the same setting to SH_MEM_CONFIG and VM_CONTEXT1_CNTL. This
makes the noretry param no longer KFD-specific. On GFX10 I'm not
changing SH_MEM_CONFIG in this commit because GFX10 has different
retry behaviour in the SQ and I don't have a way to test it at the
moment.
Suggested-by: NChristian König <Christian.Koenig@amd.com>
CC: Philip Yang <Philip.Yang@amd.com>
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by : Shaoyun.liu < Shaoyun.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

kmemdup is introduced to duplicate a region of memory in a neat way.
Rather than kmalloc/kzalloc + memcpy, which the programmer needs to
write the size twice (sometimes lead to mistakes), kmemdup improves
readability, leads to smaller code and also reduce the chances of mistakes.
Suggestion to use kmemdup rather than using kmalloc/kzalloc + memcpy.
Reviewed-by: NChristian König <Christian.Koenig@amd.com>
Reviewed-by: NEmil Velikov <emil.velikov@collabora.com>
Signed-off-by: NFuqian Huang <huangfq.daxian@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Recommended by the hw team.
Reviewed-and-Tested-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Recommended by the hw team.
Reviewed-and-Tested-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Navi10 has 2 gfx pipe and need to enable gfx eop interrupt
per pipe, instead of enable eop int for all gfx pipes at one
time.
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The function now will create mqd bos for both gfx queue and compute queue
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NJack Xiao <jack.xiao@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Acked-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Similar to what we do for compute already.
Signed-off-by: NHawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: NJack Xiao <jack.xiao@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Acked-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Explicitly set mmGDS_VMID0_BASE to 0. Also update
GDS_VMID0_BASE/_SIZE with direct register writes.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When calling debugfs functions, there is no need to ever check the
return value.  The function can work or not, but the code logic should
never do something different based on this.

Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: "David (ChunMing) Zhou" <David1.Zhou@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: xinhui pan <xinhui.pan@amd.com>
Cc: Evan Quan <evan.quan@amd.com>
Cc: Feifei Xu <Feifei.Xu@amd.com>
Cc: amd-gfx@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Since Hardware bug, GDS exist ECC error after cold boot up,
adding GDS clearing workaround in later init for gfx9.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Use SMU firmware version to indentify the raven1 refresh device and
then load homologous RLC FW.
Signed-off-by: NPrike Liang <Prike.Liang@amd.com>
Suggested-by: Huang Rui<Ray.Huang@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

EDC counts are related to instance and se. They are not the same
for different type of EDC. EDC clearing are changed to base on
individual EDC's instance and SE number.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

With this commit drm/amd/ has no longer any uses of
the deprecated drmP.h header file.
Signed-off-by: NSam Ravnborg <sam@ravnborg.org>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: "David (ChunMing) Zhou" <David1.Zhou@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20190609220757.10862-11-sam@ravnborg.org

Fix so no files in drm/amd/ depends on the
deprecated drm_os_linux.h header file.

It was done manually:
- remove drm_os_linux.h from drmP.h
- fix all build errros
Signed-off-by: NSam Ravnborg <sam@ravnborg.org>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: "Christian König" <christian.koenig@amd.com>
Cc: "David (ChunMing) Zhou" <David1.Zhou@amd.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Daniel Vetter <daniel@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/msgid/20190609220757.10862-3-sam@ravnborg.org

Use SMU firmware version to indentify the raven1 refresh device and
then load homologous RLC FW.
Signed-off-by: NPrike Liang <Prike.Liang@amd.com>
Suggested-by: Huang Rui<Ray.Huang@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For the VF, some registers only could be programmed with RLC.
Signed-off-by: NEmily Deng <Emily.Deng@amd.com>
Reviewed-by: NTrigger Huang <Trigger.Huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

As it will destroy clear_state_obj, and also will unpin it in the
gfx_v9_0_sw_fini, so don't need to
call amdgpu_bo_free_kernel in gfx_v9_0_sw_fini, or it will have unpin warning.
Signed-off-by: NEmily Deng <Emily.Deng@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

As far as we know this was never used by userspace and so should be removed.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NMarek Olšák <marek.olsak@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

During S3/S4 bootloader will re-init ras state behind us.
Resume might fail or raise a gpu reset.
Signed-off-by: Nxinhui pan <xinhui.pan@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJames Zhu <James.Zhu@amd.com>
Tested-by: NJames Zhu <James.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

During S3 test, when system wake up and resume, ras interface
is already allocated. Move workaround before ras jumps to resume
step in gfx_v9_0_ecc_late_init, and make sure workaround applied
during resume. Also remove unused mmGB_EDC_MODE clearing.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Under Vega10 SR-IOV, with new RLC's new feature, VF should call RLC
to program some registers if supported
Signed-off-by: NTrigger Huang <Trigger.Huang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For Vega10 SR-IOV VF, skip setting some regs due to:
1, host will program them
2, avoid VF register programming violations
Signed-off-by: NTrigger Huang <Trigger.Huang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

request a gpu reset if ras return EAGAIN.
we will run late init again so it is ok to do nothing this time.
Signed-off-by: Nxinhui pan <xinhui.pan@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

When RAS is enabled, initializes the VGPRs/LDS/SGPRs and
resets EDC error counts. This is done in late_init, before
RAS TA GFX enable.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

IP initialize ras in late_init, because of the BUGs of PSP or any
other components, driver receives unexpected irqs. It is ok to add such
check anyway.
Signed-off-by: Nxinhui pan <xinhui.pan@amd.com>
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

handle ras enable on boot.
Signed-off-by: Nxinhui pan <xinhui.pan@amd.com>
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The rlc reset function is not necessary during gfx9 initialization/resume phase.
And this function would even cause rlc fw loading failed on some gfx9 ASIC.
Remove this function safely with verification well on Vega/Raven platform.
Signed-off-by: NLe Ma <le.ma@amd.com>
Reviewed-by: NFeifei Xu <Feifei.Xu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

RLC #53815 ucode has the noise issue on 4k playback while gfxoff enabled.
Signed-off-by: NHuang Rui <ray.huang@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Tested-by: NTom St Denis <tom.stdenis@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This patch enables gfxoff and stutter mode again, since we take more testing on
raven series. For raven2 and picasso, we can enable it directly. And for raven,
we need check the RLC/SMC ucode version cannot be less than #531/0x1e45.

v2: add smc version checking for raven.
Signed-off-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Tested-by: Likun Gao <Likun.Gao@amd.com> (v2)
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Suspend will put irq, so resume need get irq back.
And in the same time, skip other ras initialization.
Signed-off-by: Nxinhui pan <xinhui.pan@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

RAS ECC event will combine with GPU reset event, due to
ECC interrupts are caused by uncorrectable error that triggers
GPU reset.

v2: Fix misleading-indentation warning
v3: fix build with CONFIG_HSA_AMD disabled
Signed-off-by: NEric Huang <JinhuiEric.Huang@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Register ecc interrupts and ecc interrupt handler on gfx9.
Add ras support on gfx9

v2: squash in warning fix
Signed-off-by: NFeifei Xu <Feifei.Xu@amd.com>
Signed-off-by: Nxinhui pan <xinhui.pan@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

GC owner suggested the setting should be applied which is missed by HW default
Signed-off-by: Nshaoyunl <shaoyun.liu@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The original change caused a regression, so revert it until the new fix
is ready.

BUG: https://bugs.freedesktop.org/show_bug.cgi?id=109650

This reverts commit 764c85fef41722db0f21558c6c2fb38bee172d19.
Signed-off-by: NYong Zhao <Yong.Zhao@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

CP_RB_DOORBELL_RANGE_LOWER/UPPER and CP_MEC_DOORBELL_RANGE_LOWER/UPPER
are used for waking up an idle scheduler and for power gating support.
Usually the first few doorbells in pci doorbell bar are used for RB
and all leftover for MEC. This patch fixes the incorrect settings.

Theoretically, gfx ring doorbells should come before all MEC doorbells
to be consistent with the design. However, since the doorbell
allocations are agreed by all and we are not free to change them, also
considering the kernel MEC ring doorbells which are before gfx ring
doorbells are not used often, we compromise by leaving the doorbell
allocations unchanged.
Signed-off-by: NYong Zhao <Yong.Zhao@amd.com>
Reviewed-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

I'm not increasing the DRM version because GDS isn't totally without bugs yet.

v2: update emit_ib_size
Signed-off-by: NMarek Olšák <marek.olsak@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Replace the last bool type parameter with a general flags parameter,
to make the last parameter be able to contain more information.

v2: drop setting need_ctx_switch = false
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NHawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: NJack Xiao <Jack.Xiao@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>