The buffer object backing the user fence is reserved using the non-user
fence, i.e., as soon as the non-user fence is signaled, the user fence
buffer object can be moved or even destroyed.

Therefore, emit the user fence first.

Both fences have the same cache invalidation behavior, so this should
have no user-visible effect.
Signed-off-by: NNicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Was missed when updating the uvd 6 module.

Fixes: 1aac3c91 (drm/amdgpu: fix insert nop for UVD6 ring)
Reviewed-by: NLeo Liu <leo.liu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

We still use legacy type of gem_object_create, it should update to ttm_bo_type
now.
Signed-off-by: NHuang Rui <ray.huang@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This is dual licensed under GPL-2.0 or MIT.
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NDirk Hohndel (VMware) <dirk@hohndel.org>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This patch unifies the naming of DRM functions for reference counting
of struct drm_device. The resulting code is more aligned with the rest
of the Linux kernel interfaces.
Signed-off-by: NThomas Zimmermann <contact@tzimmermann.org>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Set the me instance in early init and use that rather than
calculating the instance based on the ring pointer.
Reviewed-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NLeo Liu <leo.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Set the me instance in early init and use that rather than
calculating the instance based on the ring pointer.
Reviewed-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NLeo Liu <leo.liu@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

new_active_crtcs is a bitmask, new_active_crtc_count is the
actual count.
Reviewed-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The phys and vm versions had the values swapped.
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The parameter was removed.

Fixes: a19741e5 "dma_buf: remove device parameter from attach
                     callback v2"
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

To hopefully make the code dealing with GPU vs CPU pages a little
clearer.
Suggested-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The other day I was testing one of the HP laptops at my office with an
i915/amdgpu hybrid setup and noticed that hotplugging was non-functional
on almost all of the display outputs. I eventually discovered that all
of the external outputs were connected to the amdgpu device instead of
i915, and that the hotplugs weren't being detected so long as the GPU
was in runtime suspend. After some talking with folks at AMD, I learned
that amdgpu is actually supposed to support hotplug detection in runtime
suspend so long as the OEM has implemented it properly in the firmware.

On this HP ZBook 15 G4 (the machine in question), amdgpu wasn't managing
to find the ATIF handle at all despite the fact that I could see acpi
events being sent in response to any hotplugging. After going through
dumps of the firmware, I discovered that this machine did in fact
support ATIF, but that it's ATIF method lived in an entirely different
namespace than this device's handle (the device handle was
\_SB_.PCI0.PEG0.PEGP, but ATIF lives in ATPX's handle at
\_SB_.PCI0.GFX0).

So, fix this by probing ATPX's ACPI parent's namespace if we can't find
ATIF elsewhere, along with storing a pointer to the proper handle to use
for ATIF and using that instead of the device's handle.

This fixes HPD detection while in runtime suspend for this ZBook!

v2: Update the comment to reflect how the namespaces are arranged
based on the system configuration. (Alex)
Signed-off-by: NLyude Paul <lyude@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Since it seems that some vendors are storing the ATIF ACPI methods under
the same handle that ATPX lives under instead of the device's own
handle, we're going to need to be able to retrieve this handle later so
we can probe for ATIF there.
Signed-off-by: NLyude Paul <lyude@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Fix typo.
Reviewed-by: NJim Qu <Jim.Qu@amd.com>
Signed-off-by: NLyude Paul <lyude@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Currently, there is nothing in amdgpu that actually uses these structs
other than amdgpu_acpi.c. Additionally, since we're about to start
saving the correct ACPI handle to use for calling ATIF in this struct
this saves us from having to handle making sure that the acpi_handle
(and by proxy, the type definition for acpi_handle and all of the other
acpi headers) doesn't need to be included within the amdgpu_drv struct
itself. This follows the example set by amdgpu_atpx_handler.c.
Signed-off-by: NLyude Paul <lyude@redhat.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable stutter mode can save power in low DRAM use cases
including but not limited to productivity application use,
web browsing, and video playback.

Currently this feature is disabled by default.

Make bit 17 in module parameter amdgpu_pp_feature_mask
as stutter mode mask, so user can enable/disable this feature easily.
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NHuang Rui <ray.huang@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Partially revert commit 2dc80b00
("drm/amdgpu: optimize amdgpu driver load & resume time")'

1. CG/PG enablement are part of gpu hw ip initialize, we should
wait for them complete. otherwise, there are some potential conflicts,
for example, Suspend and CG enablement concurrently.
2. better run ib test after hw initialize completely. That is to say,
   ib test should be after CG/PG enablement. otherwise, the test will
   not cover the cg/pg/poweroff enable case.
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

1. add amdgpu_device_ip_late_set_pg_state function for
   set pg state.
2. delete duplicate pg state setting on gfx_v8_0's late_init.
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

we can take gfx off feature as gfx power gate. gfx off feature is also
controled by smu. so add gfx_off support in pp_set_powergating_by_smu.
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

gfx ip block can call set_powergating_by_smu to set gfx pg state if
necessary.
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Some HW ip blocks need call SMU to enter/leave power gate state.
So export common set_powergating_by_smu interface.

1. keep consistent with set_clockgating_by_smu
2. scales easily to powergate other ip(gfx) if necessary
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

In order to keep consistent with powergate_uvd/vce.
Reviewed-by: NEvan Quan <evan.quan@amd.com>
Signed-off-by: NRex Zhu <Rex.Zhu@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

the fence_context and seqno is init in amdgpu_vm_manager_init() &
amdgpu_vmid_mgr_init(). remove the amdgpu_vmid_mgr_init() copy.
Signed-off-by: NFlora Cui <Flora.Cui@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: NChunming Zhou <david1.zhou@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Everything in the flush code path (i.e. waiting for SW queue
to become empty) names with *_flush()
and everything in the release code path names *_fini()
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Suggested-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Everything in the flush code path (i.e. waiting for SW queue
to become empty) names with *_flush()
and everything in the release code path names *_fini()

This patch also effect the amdgpu and etnaviv drivers which
use those functions.

v2:
Also pplay the change to vd3.
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Suggested-by: NChristian König <christian.koenig@amd.com>
Acked-by: NLucas Stach <l.stach@pengutronix.de>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This reverts commit 8f4805a2.

This change was mistakenly committed and should have been dropped
with the rest of the series.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Access to SQ_EDC_INFO requires selecting register instance and
hence mutex lock when accessing GRBM_GFX_INDEX for which a work
is schedueled from IH. But SQ interrupt can be raised on many instances
at once which means queuing work will usually succeed for the first one
but fail for the rest since the work takes time to process. To avoid
losing info about other interrupt instances call the parsing function
directly from high IRQ when current work hasn't finished and avoid
accessing SQ_EDC_INFO in that case.

v2:
Simplify high IRQ and BH handlers synchronization using work_pending.
Remove {READ,WRITE}_ONCE notations since smp_{r,w}mb are implicit
compiler barriers.

v3:
Remove exlicit memory barriers as scedule_work has r/w barriers.
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Switch to using reg fields defines istead of magic values.
Add SH_ID and PRIV fields reading for instr. and err cases.
Signed-off-by: NAndrey Grodzovsky <andrey.grodzovsky@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This adds what should be a stable interface to read GPU
load from userspace.

(v2): Fix comments and name of file per recommendations.
(v3): Add chapter to amdgpu.rst as well.
Signed-off-by: NTom St Denis <tom.stdenis@amd.com>
Acked-by: NSlava Abramov <slava.abramov@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Always validating the VM PTs takes to much time. Only always validate
the per VM BOs for now.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NJunwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Support new VCN FW version naming convention:
  [31, 28] for VEP interface major version if applicable
  [27, 24] for decode interface major version
  [23, 20] for encode interface major version
  [19, 12] for encode interface minor version
  [11, 0]  for firmware revision
Bit 20-23, it is encode major and non-zero for new naming convention.
This field is part of version minor and DRM_DISABLED_FLAG in old naming
convention. Since the latest version minor is 0x5B and DRM_DISABLED_FLAG
is zero in old naming convention, this field is always zero so far.
These four bits are used to tell which naming convention is present.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NFang, Peter <Peter.Fang@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Here is the UBSAN dump:

[    3.866656] index 2 is out of range for type 'amdgpu_uvd_inst [2]'
[    3.866693] Workqueue: events work_for_cpu_fn
[    3.866702] Call Trace:
[    3.866710]  dump_stack+0x85/0xc5
[    3.866719]  ubsan_epilogue+0x9/0x40
[    3.866727]  __ubsan_handle_out_of_bounds+0x89/0x90
[    3.866737]  ? rcu_read_lock_sched_held+0x58/0x60
[    3.866746]  ? __kmalloc+0x26c/0x2d0
[    3.866846]  amdgpu_fence_driver_start_ring+0x259/0x280 [amdgpu]
[    3.866896]  amdgpu_ring_init+0x12c/0x710 [amdgpu]
[    3.866906]  ? sprintf+0x42/0x50
[    3.866956]  amdgpu_gfx_kiq_init_ring+0x1bc/0x3a0 [amdgpu]
[    3.867009]  gfx_v8_0_sw_init+0x1ad3/0x2360 [amdgpu]
[    3.867062]  ? smu7_init+0xec/0x160 [amdgpu]
[    3.867109]  amdgpu_device_init+0x112c/0x1dc0 [amdgpu]

'ring->me' might be set as 2 with 'amdgpu_gfx_kiq_init_ring', that would
cause out of range for 'amdgpu_uvd_inst[2]'.

v2: simplified with ring type
Signed-off-by: NLeo Liu <leo.liu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

start / last / max_entries are numbers of GPU pages, pfn / count are
numbers of CPU pages. Convert between them accordingly.

Fixes badness on systems with > 4K page size.

Cc: stable@vger.kernel.org
Bugzilla: https://bugs.freedesktop.org/106258Reported-by: NMatt Corallo <freedesktop@bluematt.me>
Tested-by: foxbat@ruin.net
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Neither used nor correctly implemented anywhere. Just completely remove
the interface.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: NSumit Semwal <sumit.semwal@linaro.org>
Link: https://patchwork.freedesktop.org/patch/226645/

The device parameter is completely unused because it is available in the
attachment structure as well.

v2: fix kerneldoc as well
Signed-off-by: NChristian König <christian.koenig@amd.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: https://patchwork.freedesktop.org/patch/226643/

Even BOs with AMDGPU_GEM_CREATE_NO_CPU_ACCESS may end up at least
partially in CPU visible VRAM, in particular when all VRAM is visible.

v2:
* Don't take VRAM mgr spinlock, not needed (Christian König)
* Make loop logic simpler and clearer.

Cc: stable@vger.kernel.org
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Preparation for the following fix, no functional change intended.

Cc: stable@vger.kernel.org
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

At least in theory, ttm_bo_validate may move the BO, in which case the
pin_size accounting would be inconsistent with when the BO was pinned.

Cc: stable@vger.kernel.org
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

All UVD instanses have only one dpm control, so it is better
to share one idle_work handle.
Signed-off-by: NJames Zhu <James.Zhu@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Tested-by: NStefan Agner <stefan@agner.ch>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>