some registers are PF & VF copy, and we can safely use
mmio method to access them.

and sometime we are forbid to use kiq to access registers
for example in INTR context.

we need a MACRO that always disable KIQ for regs accessing
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NXiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable/disable the handling globally for now and
print a warning when we enable it for the first time.

v2: write to the correct register, adjust bits to that hw generation
v3: fix compilation, add the missing register bit definitions
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Till GFX8 we can only enable PRT support globally, but with the next hardware
generation we can do this on a per page basis.

Keep the interface consistent by adding PRT mappings and enable
support globally on current hardware when the first mapping is made.

v2: disable PRT support delayed and on all error paths
v3: PRT and other permissions are mutal exclusive,
    PRT mappings don't need a BO.
v4: update PRT mappings durign CS as well, make va_flags 64bit
Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Future hardware generations can handle PRT flags on a per page basis,
but current hardware can only turn it on globally.

Add the basic handling for both, a global callback to enable/disable
triggered by setting a per mapping flag.

v2: agd: rebase fixes
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Check whether we need to post rather than whether the asic is
posted.  There are some cases (e.g., GPU reset or resume from
hibernate) where we need to force post even if the asic has
been posted.
Signed-off-by: NJim Qu <Jim.Qu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Like ttm_bo_validate(), ttm_bo_init() might need to move BO and
the number of bytes moved by TTM should be reported. This can help
the throttle buffer migration mechanism to make a better decision.

v2: fix computation
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NSamuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

They are 64bit not 32 for a while now.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

To determine whether the context uses GPUVM or not.
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Write multiple dwords to the ring.
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The scratch regs are sequential so there's no need to keep
them in an array, we can just return the index of the first
free register + the base register. Also change the array
of bools for keeping track of the free regs to a bitfield.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NNils Wallménius <nils.wallmenius@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

During virtual runtime, need to send command to kiq ring to
read/write GPU registers. Add two interface to support the two
actions.
Signed-off-by: NXiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: NMonk Linu <Monk.Liu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Call detection function driectly, so remove the interface.

V2: ci and si also need to call the detect function.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NMonk Liu <monk.liu@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NXiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Use acronym to rename fields to make easy to spell out.
Signed-off-by: NXiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

KIQ is queue-memory based initialization method: setup KIQ queue
firstly, then send command to KIQ to setup other queues, without
accessing registers.

For virtualization, need KIQ to access virtual function registers
when running on guest mode.

V2: use amdgpu_bo_create/free_kernel to allocate BO.
Signed-off-by: NMonk Liu <Monk.Liu@amd.com>
Signed-off-by: NXiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is a left over from radeon, amdgpu doesn't support any
non-atombios parts and amdgpu_device_init would bail if the
check for atombios failed anyway.
Reviewed-by: NEdward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: NNils Wallménius <nils.wallmenius@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

drm_debugfs_cleanup() now removes all minor->debugfs_list entries
automatically, so no need to call drm_debugfs_remove_files().
Also remove empty drm_driver.debugfs_cleanup callback.

Cc: alexander.deucher@amd.com
Cc: christian.koenig@amd.com
Signed-off-by: NNoralf Trønnes <noralf@tronnes.org>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20170126225621.12314-5-noralf@tronnes.org

The integer returned by the unload hook is ignored by the drm core, so
let's make it void.

This patch was created using the following Coccinelle semantic script
(except for the declaration and comment in drm_drv.h):

Compile-tested only.

// <smpl>
@ get_name @
struct drm_driver drv;
identifier fn;
@@
drv.unload = fn;

@ replace_type @
identifier get_name.fn;
@@
- int
+ void
fn (...)
{
...
}

@ remove_return_param @
identifier get_name.fn;
@@
void fn (...)
{
<...
if (...)
return
- ...
;
...>
 }

@ drop_final_return @
identifier get_name.fn;
@@
void fn (...)
{
...

- return 0;
}
// </smpl>
Suggested-by: NDaniel Vetter <daniel.vetter@intel.com>
Signed-off-by: NGabriel Krisman Bertazi <krisman@collabora.co.uk>
Acked-by: Christian König <christian.koenig@amd.com>.
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20170106175731.29196-1-krisman@collabora.co.uk

Implemented for SGPRs for GFX v8 initially.

(v2) cleanup minor whitespace and remove sanity check and
     addressing is in dwords not bytes
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Acked-by: NEdward O'Callaghan <funfunctor@folklore1984.net>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It's necessary if we want to export vbios image out.
Reviewed-by: NEdward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian Koenig <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We can't just reuse pci_remove as there may be userspace still
doing things.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=98638
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=97980Reviewed-by: NChristian König <christian.koenig@amd.com>
Reported-and-tested-by: NMike Lothian <mike@fireburn.co.uk>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This reverts commit a693e050.

This breaks on systems with huge amounts of system memory as we do not have
enough vram to store the page tables.  Additionally, this is less of an issue
with the recent gtt manager changes.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Not used.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

External clients which import our bo's wait only
for exclusive dmabuf-fences, not on shared ones,
ditto for bo's which we import from external
providers and write to.

Therefore attach exclusive fences on prime shared buffers
if our exported buffer gets imported by an external
client, or if we import a buffer from an external
exporter.

See discussion in thread:
https://lists.freedesktop.org/archives/dri-devel/2016-October/122370.html

Prime export tested on Intel iGPU + AMD Tonga dGPU as
DRI3/Present Prime render offload, and with the Tonga
standalone as primary gpu.

v2: Add a wait for all shared fences before prime export,
    as suggested by Christian Koenig.

v3: - Mark buffer prime_exported in amdgpu_gem_prime_pin,
    so we only use the exclusive fence when exporting a
    bo to external clients like a separate iGPU, but not
    when exporting/importing from/to ourselves as part of
    regular DRI3 fd passing.

    - Propagate failure of reservation_object_wait_rcu back
    to caller.

v4: - Switch to a prime_shared_count counter instead of a
      flag, which gets in/decremented on prime_pin/unpin, so
      we can switch back to shared fences if all clients
      detach from our exported bo.

    - Also switch to exclusive fence for prime imported bo's.

v5: - Drop lret, instead use int ret -> long ret, as proposed
      by Christian.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95472
Tested-by: Mike Lothian <mike@fireburn.co.uk> (v1)
Signed-off-by: NMario Kleiner <mario.kleiner.de@gmail.com>
Reviewed-by: Christian König <christian.koenig@amd.com>.
Cc: Christian König <christian.koenig@amd.com>
Cc: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

v2: agd: rebase and squash in all the previous optimizations and
changes so everything compiles.
v3: squash in Slava's 32bit build fix
v4: rebase on drm-next (fence -> dma_fence),
    squash in Monk's ioctl update patch
Signed-off-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NMonk Liu <monk.liu@amd.com>
Reviewed-by: NJammy Zhou <Jammy.Zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NSumit Semwal <sumit.semwal@linaro.org>
 [sumits: fix checkpatch warnings]
Link: http://patchwork.freedesktop.org/patch/msgid/1478290570-30982-2-git-send-email-alexander.deucher@amd.com

Move IP version specific code into a callback.

Also add support for gfx7 devices.
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This makes it easier to replace specific IP blocks on
asics for handling virtual_dce, DAL, etc. and for building
IP lists for hw or tables.  This also stored the status
information in the same structure.

v2: split out spelling fix into a separate patch
    add a function to add IPs to the list
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

No intended functional change.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

I should have suggested that on the initial patchset. This saves us a
few CPU cycles during CS and a bunch of loc.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We need to cache some additional values to handle SR-IOV
and PG.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Only cleanup, no intended functional change.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Only cleanup, no intended functional change.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Only cleanup, no intended functional change.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Saves us a bit of memory.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>