Before power gate plane, mpcc idle wait is processed,
no need to wait another time.
Signed-off-by: NYongqiang Sun <yongqiang.sun@amd.com>
Reviewed-by: NTony Cheng <Tony.Cheng@amd.com>
Acked-by: NHarry Wentland <harry.wentland@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NRoger He <Hongbo.He@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

RV implementation was missing these callbacks.  Used
to fetch the clock values for other components.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

No longer used.
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The expectation is that the base driver doesn't mess with these.
Some components interact with these directly so let the components
handle these directly.
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The expectation is that the base driver doesn't mess with these.
Some components interact with these directly so let the components
handle these directly.
Reviewed-by: NHarry Wentland <harry.wentland@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Leftover from bring up.
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The golden register arrays were empty so the function was
effectively useless.
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

They are not used any longer.  We get the scratch register
locations from the vbios directly now.
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Cleans up and consolidates all of the per-asic logic.

v2: squash in "drm/amdgpu: fix NULL err for sriov detect" (Chunming)
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

All functions should have nbio_v* prefix.
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

this fix the VCE world switch hang issue
Signed-off-by: NFrank Min <Frank.Min@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Should be 0.
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We are polling so no need for INT.
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

this VRAM evict is not needed and also cost 2seconds
to finish because the IRQ is software side disabled
before it.
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

fix calculation.
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

shrink_pages is in unit of Order after ttm_page_pool_free,
but it is used by nr_free in next round so need change
it into native page unit
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NRoger He <Hongbo.He@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

And also provide the level for which we need a PDE.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Now instead of one submission for each PDE batch them together over all
PDs who need an update.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

No more double house keeping.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Not needed any more.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We can do this with a simple mask as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

On Tonga VF, there're 2 sources updating wptr registers for
sdma3: 1) polling mem and 2) doorbell. When doorbell and polling
mem are both enabled on sdma3, there will be collision hit in
occasion between those two sources when ucode and h/w are doing
the updating on wptr register in parallel. Issue doesn't happen
on CP GFX/Compute since CP drops all doorbell writes when VF is
inactive. So enable polling mem and don't use doorbell for SDMA3.
Signed-off-by: NPixel Ding <Pixel.Ding@amd.com>
Reviewed-by: NMonk Liu <monk.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Horrible inefficient, but avoids problems when the root PD size becomes
to big.

v2: remove incr as well.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <davdi1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

That doesn't hit any more most of the time anyway.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Torture test for MM and VM support, can be used to evict all VRAM while
the system is under load.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Remove the superflous .debugfs_init callback and register all files in
amdgpu_device.c in just one function.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Remove the header where it's not used.
Acked-by: NChristian Konig <christian.koenig@amd.com>
Signed-off-by: NShaoyun Liu <Shaoyun.Liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Acked-by: NChristian Konig <christian.koenig@amd.com>
Signed-off-by: NShaoyun Liu <Shaoyun.Liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Handle dynamic offsets correctly in static arrays.
Acked-by: NChristian Konig <christian.koenig@amd.com>
Signed-off-by: NShaoyun Liu <Shaoyun.Liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Update the register access macros and functions to take into
account the new dynamic IP base offsets.
Acked-by: NChristian Konig <christian.koenig@amd.com>
Signed-off-by: NShaoyun Liu <Shaoyun.Liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The base offsets of the IP blocks may change across
asics even though the relative register offsets
are the same for an IP.  Handle this dynamically.
Acked-by: NChristian Konig <christian.koenig@amd.com>
Signed-off-by: NShaoyun Liu <Shaoyun.Liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Was missing before.
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is the only part of the scheduler which must not be called from
different drivers. Move it to module init/exit so it is done a single
time when loading the scheduler.
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Tested-by: NDieter Nützel <Dieter@nuetzel-hh.de>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This moves and renames the AMDGPU scheduler to a common location in DRM
in order to facilitate re-use by other drivers. This is mostly a straight
forward rename with no code changes.

One notable exception is the function to_drm_sched_fence(), which is no
longer a inline header function to avoid the need to export the
drm_sched_fence_ops_scheduled and drm_sched_fence_ops_finished structures.
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Tested-by: NDieter Nützel <Dieter@nuetzel-hh.de>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

These files were missing it before.
Acked-by: NHarry Wentland <harry.wentland@amd.com>
Acked-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Was missing license text.
Acked-by: NHarry Wentland <harry.wentland@amd.com>
Acked-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>