Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Update to the newer programming model.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Used for GPU clock counter snapshots.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: update to latest driver changes
v3: properly tear down vm on suspend
v4: fix up irq init ordering
v5: remove outdated comment
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

Async page table updates using the sDMA engine.  sDMA has a
special packet for updating entries for contiguous pages
that reduces overhead.

v2: add support for and use the CP for now.
v3: update for 2 level PTs
v4: rebase, fix DMA packet
v5: switch to using an IB
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Update the page table base address and flush the
VM TLB using the sDMA.

V2: update for 2 level PTs
V3: update vm flush
V4: update SH_MEM* regs
V5: switch back to old style VM TLB invalidate
V6: fix packet formatting
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

CIK has new asynchronous DMA engines called sDMA
(system DMA).  Each engine supports 1 ring buffer
for kernel and gfx and 2 userspace queues for compute.

TODO: fill in the compute setup.

v2: update to the latest reset code
v3: remove ib_parse
v4: fix copy_dma()
v5: drop WIP compute sDMA queues
v6: rebase
v7: endian fixes for IB
v8: cleanup for release
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Todo:
- handle interrupts for compute queues

v2: add documentation
v3: update to latest reset code
v4: update to latest illegal CP handling
v5: fix missing break in interrupt handler switch statement
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

RLC handles the interrupt controller and other tasks
on the GPU.

v2: add documentation
v3: update programming sequence
v4: additional setup
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Update the page table base address and flush the
VM TLB using the CP.

v2: update for 2 level PTs
v3: use new packet for invalidate
v4: update SH_MEM* regs when flushing the VM
v5: add pfp sync, go back to old style vm TLB invalidate
v6: fix hdp flush packet count
v7: use old style HDP flush
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: add documenation
v3: update the latest ib changes
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

For gfx ring only.  Compute is still todo.

v2: add documentation
v3: update to latest reset changes, integrate emit update patch.
v4: fix count on wait_reg_mem for HDP flush
v5: use old hdp flush method for fence
v6: set valid bit for IB
v7: cleanup for release
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Sets up the GFX ring and loads ucode for GFX and Compute.

Todo:
- handle compute queue setup.

v2: add documentation
v3: integrate with latest reset changes
v4: additional init fixes
v5: scratch reg write back no longer supported on CIK
v6: properly set CP_RB0_BASE_HI
v7: rebase
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Load the GDDR5 ucode and train the links.

v2: update ucode
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Currently the driver required 6 sets of ucode:
1. pfp - pre-fetch parser, part of the GFX CP
2. me - micro engine, part of the GFX CP
3. ce - constant engine, part of the GFX CP
4. rlc - interrupt, etc. controller
5. mc - memory controller (discrete cards only)
6. mec - compute engines, part of Compute CP

V2: add documentation
V3: update MC ucode
V4: rebase
V5: update mc ucode
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Redirect invalid memory accesses to the default page
instead of locking up the memory controller.

v2: rebase on top of 2 level PTs
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The vm callbacks are the same as the SI ones right now
(same regs and bits). We could share the SI variants, and
I may yet do that, but I figured I would add CIK specific
ones for now in case we need to change anything.

V2: add documentation, minor fixes.
V3: integrate vram offset fixes for APUs
V4: enable 2 level VM PTs
V5: index SH_MEM_* regs properly
V6: add ib_parse()
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: split soft reset into compute and gfx.  Still need
to make reset more fine grained, but this should be a
start.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

v2: tiling fixes
v3: more tiling fixes
v4: more tiling fixes
v5: additional register init
v6: rebase
v7: fix gb_addr_config for KV/KB
v8: drop wip KV bits for now, add missing config reg
v9: fix cu count on Bonaire
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>