Add PG lock support as well as bank selection to
the MMIO write function.
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Allow any of the se/sh/instance fields to be
specified as a broadcast by submitting 0x3FF.

(v2) Fix broadcast range checking
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

On non VI/CZ platforms it would not free
the grbm index lock.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Currently supports CZ/VI.  Allows nearly atomic read
of wave data from GPU.
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This makes it easier to replace specific IP blocks on
asics for handling virtual_dce, DAL, etc. and for building
IP lists for hw or tables.  This also stored the status
information in the same structure.

v2: split out spelling fix into a separate patch
    add a function to add IPs to the list
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Update the comment to explain why we do this.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Split VRAM allocations into 4MB blocks.

v2: fix typo in comment, some suggested cleanups
v3: document how to disable the feature, fix rebase issue
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NEdward O'Callaghan <funfunctor@folklore1984.net>
Tested-by: NMike Lothian <mike@fireburn.co.uk>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Add a flag noting that a BO must be created using linear VRAM
and set this flag on all in kernel users where appropriate.

Hopefully I haven't missed anything.

v2: add it in a few more places, fix CPU mapping.
v3: rename to VRAM_CONTIGUOUS, fix typo in CS code.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NEdward O'Callaghan <funfunctor@folklore1984.net>
Tested-by: NMike Lothian <mike@fireburn.co.uk>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable multi crtcs for virtual display, user can set the number of crtcs
by amdgpu module parameter  virtual_display.

v2: make timers per crtc
v3: agd: simplify implementation
Signed-off-by: NEmily Deng <Emily.Deng@amd.com>
Reviewed-By: NEmily Deng <Emily.Deng@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

to match the other functions in that file.
Reviewed-By: NEmily Deng <Emily.Deng@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Otherwise, you can't disable dpm.

Tested-by and Reviewed-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

IP types are not an index.  Each asic may have number and
type of IPs.  Properly check the the type rather than
using the type id as an index.

v2: fix all the IPs to not use IP type as an idx as well.
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

UVD and VCE CG are handled specially, however the previous
fix for this skipped late init for those blocks rather than
just CG.  Just protect the CG function call.  No functional
change since UVD and VCE don't currently utilize a late_init
function.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Sometimes during multiple reboots, the system hangs
during bootup. The issue is very random and happens
once in around 50 reboots or so.

It seems if clockgating is enabled before late init,
the GFX engine sometimes does not respond.

This patch changes the ordering a little so that
both powergating and clockgating are enabled only
after late init calls.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NArindam Nath <arindam.nath@amd.com>
Tested-by: NSunil Uttarwar <Sunil.Uttarwar1@amd.com>
Reviewed-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Successful sw_init() and hw_init() states are tracked, but not
late_init(). Various error paths may result in amdgpu_fini() being
called before .late init is done, so late_init needs to be tracked
to avoid unexpected or multiple .late_fini() calls.
Signed-off-by: NGrazvydas Ignotas <notasas@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Some code called by drm_crtc_force_disable_all() wants to wait for all
fences, so only do fence teardown after CRTCs are disabled.

Fixes: 84b89bdc ("drm/amdgpu: Turn off CRTCs on driver unload")
Cc: stable@vger.kernel.org # v4.8+
Signed-off-by: NGrazvydas Ignotas <notasas@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We get 7 warnings when building kernel with W=1:
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c:1990:5: warning: no previous prototype for 'amdgpu_pre_soft_reset' [-Wmissing-prototypes]
drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c:1548:5: warning: no previous prototype for 'amdgpu_connector_virtual_dpms' [-Wmissing-prototypes]
drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.c:1560:5: warning: no previous prototype for 'amdgpu_connector_virtual_set_property' [-Wmissing-prototypes]
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c:330:5: warning: no previous prototype for 'amdgpu_cs_list_validate' [-Wmissing-prototypes]
drivers/gpu/drm/amd/amdgpu/dce_virtual.c:98:6: warning: no previous prototype for 'dce_virtual_stop_mc_access' [-Wmissing-prototypes]
drivers/gpu/drm/amd/amdgpu/dce_virtual.c:130:6: warning: no previous prototype for 'dce_virtual_resume_mc_access' [-Wmissing-prototypes]
drivers/gpu/drm/amd/amdgpu/dce_virtual.c:136:6: warning: no previous prototype for 'dce_virtual_set_vga_render_state' [-Wmissing-prototypes]

In fact, all of the functions are only used in the file
in which they are declared and don't need a declaration,
but can be made static.

So this patch marks both functions with 'static'.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NBaoyou Xie <baoyou.xie@linaro.org>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

1,Changes on virtualization detections
2,Don't load smu & mc firmware if using sr-iov bios
3,skip vPost for sriov & force vPost if dev pass-through

v2: agd: squash in Rays's fix for the missed SI case
v3: agd: squash in additional fixes for CIK, SI, cleanup
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

A multiplication for the size determination of a memory allocation
indicated that an array data structure should be processed.
Thus use the corresponding function "kmalloc_array".

This issue was detected by using the Coccinelle software.
Signed-off-by: NMarkus Elfring <elfring@users.sourceforge.net>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

(v2) Tidy'ed up read function.
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NEdward O'Callaghan <funfunctor@folklore1984.net>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is not necessary as pointed out by Lukas Wunner.

This reverts commit f46cf373.

no need to set cg for uvd/vce in late init.
As when ring test, uvd/vce's dpm will be enabled/disabled.
the cg will be set.

fix issue suspend when play video or encode, then resume back,
the clock will be bypassed on Polaris/Fiji.
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

reset the asic if adapter is not powerdown when doing freeze()
thaw() and restore(), in order to get a valid state of adapter.

v2: squash in warning fix from Rex
Signed-off-by: NJimQu <Jim.Qu@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Tested By: Shawn Starr <shawn.starr@rogers.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Missed this case previously. No need to do anything if the
device is already off.  runtime pm will handle it.
Acked-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

V2:
a. gart recovery should be ahead of ring test.
b. rename to amdgpu_ttm_recover_gart
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

V2:
spin lock instead of mutex for gtt list
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NKen Wang <Qingqing.Wang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v3: guard doorbell_fini as well
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NKen Wang <Qingqing.Wang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NKen Wang <Qingqing.Wang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This patch adds pcie port read/write entry, because it will be also
used on si dpm part.
Acked-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The old mechanism used a per-submission limit that didn't take previous
submissions within the same time frame into account. It also filled VRAM
slowly when VRAM usage dropped due to a big eviction or buffer deallocation.

This new method establishes a configurable MBps limit that is obeyed when
VRAM usage is very high. When VRAM usage is not very high, it gives
the driver the freedom to fill it quickly. The result is more consistent
performance.

It can't keep the BO move rate low if lots of evictions are happening due
to VRAM fragmentation, or if a big buffer is being migrated.

The amdgpu.moverate parameter can be used to set a non-default limit.
Measurements can be done to find out which amdgpu.moverate setting gives
the best results.

Mainly APUs and cards with small VRAM will benefit from this. For F1 2015,
anything with 2 GB VRAM or less will benefit.

Some benchmark results - F1 2015 (Tonga 2GB):

Limit      MinFPS AvgFPS
Old code:  14     32.6
128 MB/s:  28     41
64 MB/s:   15.5   43
32 MB/s:   28.7   43.4
8 MB/s:    27.8   44.4
8 MB/s:    21.9   42.8 (different run)

Random drops in Min FPS can still occur (due to fragmented VRAM?), but
the average FPS is much better. 8 MB/s is probably a good limit for this
game & the current VRAM management. The random FPS drops are still to be
tackled.

v2: use a spinlock
Signed-off-by: NMarek Olšák <marek.olsak@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The old names were dragged over from radeon.  The new ones
better match the naming conventions used in the driver.

No functional change.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

V2:
1. don't directly submit to many jobs at the same time.
2. delete unrelated printk.
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

V2:
1. use mutex instead of spinlock for shadow list, since its process could
sleep.
2. move list_del to bo destroy phase.
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

V2:
add checking if need backup in amdgpu_bo_create.
Signed-off-by: NChunming Zhou <David1.Zhou@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Adds family and external_rev_id to config data
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Adds rev_id as well as cg/pg flags to help debug runtime.
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For virtual display feature, as there may be multiple GPUs,
for user could choose whiche GPU need to enable this feature, change
the type of virtual_display from int to char*. The variable will be set
like this virtual_display="xxxx:xx:xx.x;xxxx:xx:xx.x;".
Signed-off-by: NEmily Deng <Emily.Deng@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>