This patch fixes a bug in kgd_set_pasid_vmid_mapping(), where the function
only updated the ATC registers (IOMMU) with the new VMID <--> PASID mapping,
but didn't update the IH (Interrupt) registers.

The bug only occurs when using non-HWS mode. In HWS mode, the CP automatically
does the VMID <--> PASID mapping.
Signed-off-by: NBen Goz <ben.goz@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>
Acked-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enable smc fan control for CI boards.  Should
reduce the fan noise on systems with a higher
default fan profile.

v2: disable by default, add additional fan setup, rpm control

bug:
https://bugs.freedesktop.org/show_bug.cgi?id=73338Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This patch adds the interface between the radeon driver and the amdkfd driver.
The interface implementation is contained in radeon_kfd.c and radeon_kfd.h.

The interface itself is represented by a pointer to struct
kfd_dev. The pointer is located inside radeon_device structure.

All the register accesses that amdkfd need are done using this interface. This
allows us to avoid direct register accesses in amdkfd proper,  while also
avoiding locking between amdkfd and radeon.

The single exception is the doorbells that are used in both of the drivers.
However, because they are located in separate pci bar pages, the danger of
sharing registers between the drivers is minimal.

Having said that, we are planning to move the doorbells as well to radeon.

v3:

Add interface for sa manager init and fini. The init function will allocate a
buffer on system memory and pin it to the GART address space via the radeon sa
manager.

All mappings of buffers to GART address space are done via the radeon sa
manager. The interface of allocate memory will use the radeon sa manager to sub
allocate from the single buffer that was allocated during the init function.

Change lower_32/upper_32 calls to use linux macros

Add documentation for the interface

v4:

Change ptr field type in kgd_mem from uint32_t* to void* to match to type that
is returned by radeon_sa_bo_cpu_addr

v5:

Change format of mqd structure to work with latest KV firmware
Add support for AQL queues creation to enable working with open-source HSA
runtime.
Move generic kfd-->kgd interface and other generic kgd definitions to a generic
header file that will be used by AMD's radeon and amdgpu drivers
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>

Volatile bit was in the wrong location.  This bit is
not used at the moment.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

This patch makes it possible to decide how many address
bits are spend on the page directory vs the page tables.

v2: remove unintended change
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Testing the update pending bit directly after issuing an
update is nonsense cause depending on the pixel clock the
CRTC needs a bit of time to execute the flip even when we
are in the VBLANK period.

This is just a non invasive patch to solve the problem at
hand, a more complete and cleaner solution should follow
in the next merge window.

Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=76564

v2: fix source IDs for CRTC2-6
Signed-off-by: NChristian König <christian.koenig@amd.com>
Cc: stable@vger.kernel.org

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Only VCE 2.0 support so far.

v2: squashing multiple patches into this one
v3: add IRQ support for CIK, major cleanups,
    basic code documentation
v4: remove HAINAN from chipset list
Signed-off-by: NChristian König <christian.koenig@amd.com>

To support HSA on KV, we need to limit the number of vmids and pipes
that are available for radeon's use with KV.

This patch reserves VMIDs 8-15 for amdkfd (so radeon can only use VMIDs
0-7) and also makes radeon thinks that KV has only a single MEC with a single
pipe in it

v3: Use define for static vmid allocation in radeon
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>

pci config reset is a low level reset that resets
the entire chip from the bus interface.  It can
be more reliable if soft reset fails.

v2: fix rebase
v3: hide behind module parameter
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The formula needs to be adjusted since there are 4 RBs
per SH rather than 2 as on previous asics.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Hawaii uses a different tiling configuration.  Add support
for it.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Skip programming a register that was removed and
adjust the mask of the VM client status.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This adds the hawaii asic specific configuration
details.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Uses the CP ring rather than the DMA ring.  Useful
for debugging and benchmarking.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The FMT blocks control how data is sent from the backend
of the display pipe to to monitor.  Proper set up of the
FMT blocks are required for 30bpp formats.  Additionally,
dithering can be enabled on for better display with 18 and
24bpp displays.  The exception is LVDS/eDP which atom
takes care of in the SelectCRTC_Source table.  For now
just enable truncation until we test dithering more.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We need to allocate line buffer to each display when
setting up the watermarks.  Failure to do so can lead
to a blank screen.  This fixes blank screen problems
on dce8 asics.

Based on an initial fix from:
Jay Cornwall <jay.cornwall@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

- use new cg/pg flags for finer grained clock and
powergating control
- restructure the cg/pg code so it can be called from
other components such as dpm

v2: fix build breakage from rebase
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This allows you to look at the current DPM state via
debugfs.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This allows you to look at the current DPM state via debugfs.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This adds dpm support for btc asics. This includes:
- dynamic engine clock scaling
- dynamic memory clock scaling
- dynamic voltage scaling
- dynamic pcie gen switching

Set radeon.dpm=1 to enable.

v2: remove unused radeon_atombios.c changes,
    make missing smc ucode non-fatal
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This adds dpm support for KB/KV asics.  This includes:
- dynamic engine clock scaling
- dynamic voltage scaling
- power containment
- shader power scaling

Set radeon.dpm=1 to enable.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This added support for the on-chip thermal sensors on
CIK asics.

v2: fix register offset.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Only the APUs support power gating.

v2: disable cgcg for now
v3: workaround hw issue in mgcg
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Restructure rlc setup to handle clock and power
gating.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Enables PCIE ASPM (Active State Power Management) on
CIK asics.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Helpful for debugging GPUVM errors as we can see what
hw block and page generated the fault in the log.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

On CIK, the compute rings work slightly differently than
on previous asics, however the basic concepts are the same.

The main differences:
- New MEC engines for compute queues
- Multiple queues per MEC:
  - CI/KB: 1 MEC, 4 pipes per MEC, 8 queues per pipe = 32 queues
  -    KV: 2 MEC, 4 pipes per MEC, 8 queues per pipe = 64 queues
- Queues can be allocated and scheduled by another queue
- New doorbell aperture allows you to assign space in the aperture
  for the wptr which allows for userspace access to queues

v2: add wptr shadow, fix eop setup
v3: fix comment
v4: switch to new callback method
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>

v2: agd5f: fix clock dividers setup for bonaire
v3: agd5f: rebase
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Update to the newer programming model.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Used for GPU clock counter snapshots.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

CIK has new asynchronous DMA engines called sDMA
(system DMA).  Each engine supports 1 ring buffer
for kernel and gfx and 2 userspace queues for compute.

TODO: fill in the compute setup.

v2: update to the latest reset code
v3: remove ib_parse
v4: fix copy_dma()
v5: drop WIP compute sDMA queues
v6: rebase
v7: endian fixes for IB
v8: cleanup for release
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>