This commit makes sure that on process termination, after
we're destroying all the active queues, we're killing all the
existing wave front of the current process.

By doing this we're making sure that if any of the CUs were blocked
by infinite loop we're enforcing it to end the shader explicitly.
Signed-off-by: NBen Goz <ben.goz@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

This patch adds three new interfaces to kfd2kgd interface file of radeon.

The interfaces are:

- Check if a specific VMID has a valid PASID mapping
- Retrieve the PASID which is mapped to a specific VMID
- Issue a VMID invalidation request to the ATC
Signed-off-by: NAlexey Skidanov <Alexey.Skidanov@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

v2:

- rename get_dbgmgr_mutex to kfd_get_dbgmgr_mutex to namespace it
- change void* to uint64_t inside ioctl arguments
- use kmalloc instead of kzalloc because we use copy_from_user
  immediately after it
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

v2:

- rename get_dbgmgr_mutex to kfd_get_dbgmgr_mutex to namespace it
- change void* to uint64_t inside ioctl arguments
- use kmalloc instead of kzalloc because we use copy_from_user
  immediately after it
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

v2: rename get_dbgmgr_mutex to kfd_get_dbgmgr_mutex to namespace it
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

The address watch operation gives the ability to specify watch points
which will generate a shader breakpoint, based on a specified single
address or range of addresses.

There is support for read/write/any access modes.
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

The wave control operation supports several command types executed upon
existing wave fronts that belong to the currently debugged process.

The available commands are:

HALT   - Freeze wave front(s) execution
RESUME - Resume freezed wave front(s) execution
KILL   - Kill existing wave front(s)
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

This patch adds the skeleton H/W debugger module support. This code
enables registration and unregistration of a single HSA process at a
time.

The module saves the process's pasid and use it to verify that only the
registered process is allowed to execute debugger operations through the
kernel driver.

v2: rename get_dbgmgr_mutex to kfd_get_dbgmgr_mutex to namespace it
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

This patch adds support for static user-mode queues in QCM.
Queues which are designated as static can NOT be preempted by
the CP microcode when it is executing its scheduling algorithm.

This is needed for supporting the debugger feature, because we
can't allow the CP to preempt queues which are currently being debugged.

The number of queues that can be designated as static is limited by the
number of HQDs (Hardware Queue Descriptors).
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

This patch adds four new IOCTLs to amdkfd. These IOCTLs expose a H/W
debugger functionality to the userspace.

The IOCTLs are:

- AMDKFD_IOC_DBG_REGISTER:

The purpose of this IOCTL is to notify amdkfd that a process wants to use
GPU debugging facilities on itself only.
It is expected that this IOCTL would be called before any other H/W
debugger requests are sent to amdkfd and for each GPU where the H/W
debugging needs to be enabled. The use of this IOCTL ensures that only
one instance of a debugger is active in the system.

- AMDKFD_IOC_DBG_UNREGISTER:

This IOCTL detaches the debugger/debugged process from the H/W
Debug which was established by the AMDKFD_IOC_DBG_REGISTER IOCTL.

- AMDKFD_IOC_DBG_ADDRESS_WATCH:

This IOCTL allows to set different watchpoints with various conditions as
indicated by the IOCTL's arguments. The available number of watchpoints
is retrieved from topology. This operation is confined to the current
debugged process, which was registered through AMDKFD_IOC_DBG_REGISTER.

- AMDKFD_IOC_DBG_WAVE_CONTROL:

This IOCTL allows to control a wavefront as indicated by the IOCTL's
arguments. For example, you can halt/resume or kill either a
single wavefront or a set of wavefronts. This operation is confined to
the current debugged process, which was registered through
AMDKFD_IOC_DBG_REGISTER.

Because the arguments for the address watch IOCTL and wave control IOCTL
are dynamic, meaning that they could vary in size, the userspace passes a
pointer to a structure (in userspace) that contains the value of the
arguments. The kernel driver is responsible to parse this structure and
validate its contents.

v2: change void* to uint64_t inside ioctl arguments
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

This patch adds new interface functions to the kfd2kgd interface file. The
new functions allow to perform H/W debugger operations by writing to GPU
registers.
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Use the generic mechanism to declare a bitmap instead of unsigned long.

It seems that "struct kfd_process.allocated_queue_bitmap" is unused.
Maybe it could be deleted instead.
Signed-off-by: NJoe Perches <joe@perches.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

dma_alloc_coherent() can return memory in the vmalloc range.
virt_to_page() cannot handle such addresses and crashes. This
patch detects such cases and obtains the struct page * using
vmalloc_to_page() instead.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Acked-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

We already set this limit for the GGTT.

This is a temporary patch until a full replacement of size_t variables
(inadequate in 32-bit kernel) is in place.

Regression from:
	commit a4e0bedc
	Author: Michel Thierry <michel.thierry@intel.com>
	Date:   Wed Apr 8 12:13:35 2015 +0100

		drm/i915: Use complete address space in true PPGTT

v2: Prettify code and explain why this is needed. (Chris)
v3: Don't hide the compilation warning in 32-bit. (Chris)
Suggested-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: NMichel Thierry <michel.thierry@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

With unified modeset and flip paths introduced recently when switching
to fbcon PSR was being disabled on fb_set_par path but re-enabled on
fb_pan_display one, causing missed screen updates and un unusable
console.

Regression introduced with:

commit bb546623
Author: Ander Conselvan de Oliveira <ander.conselvan.de.oliveira@intel.com>
Date:   Tue Apr 21 17:13:13 2015 +0300

    drm/i915: Unify modeset and flip paths of intel_crtc_set_config()

Cc: Ander Conselvan de Oliveira <ander.conselvan.de.oliveira@intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Without this frontbuffer flip when enabling planes PSR got compromised
and wasn't being enabled waiting forever on the flush that never
arrived.

Another solution would to create a enable_cursor function and split this
frontbuffer flip among the different plane enable and disable functions.
But if necessary this can be done in a follow up work. For now let's
just fix the regression.

It was removed by:

commit 87d4300a
Author: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Date:   Tue Apr 21 17:12:54 2015 +0300

    drm/i915: Move intel_(pre_disable/post_enable)_primary to intel_display.c, and use it there.

Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The drm_mm debugfs output is difficult to read as two different formats
are used for the addresses:

0x00000080000000-0x0000008000b000: 45056: used
0x8000b000-0x80016000: 45056: free
0x00000080016000-0x0000008001b000: 20480: used
0x8001b000-0x817a1000: 24666112: free
0x000000817a1000-0x000000817a8000: 28672: used
0x000000817a8000-0x00000081ba8000: 4194304: used

Fix this by using %#018llx for all addresses, thus making the output:

0x0000000080000000-0x000000008000b000: 45056: used
0x000000008000b000-0x0000000080016000: 45056: free
0x0000000080016000-0x000000008001b000: 20480: used
0x000000008001b000-0x00000000817a1000: 24666112: free
0x00000000817a1000-0x00000000817a8000: 28672: used
0x00000000817a8000-0x0000000081ba8000: 4194304: used
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This patch deinlines indirect register accessor functions.

These functions perform two mmio accesses, framed by spin lock/unlock.
Spin lock/unlock by itself takes more than 50 cycles in ideal case
(if lock is exclusively cached on current CPU).

With this .config: http://busybox.net/~vda/kernel_config,
after uninlining these functions have sizes and callsite counts
as follows:

r600_uvd_ctx_rreg: 111 bytes, 4 callsites
r600_uvd_ctx_wreg: 113 bytes, 5 callsites
eg_pif_phy0_rreg: 106 bytes, 13 callsites
eg_pif_phy0_wreg: 108 bytes, 13 callsites
eg_pif_phy1_rreg: 107 bytes, 13 callsites
eg_pif_phy1_wreg: 108 bytes, 13 callsites
rv370_pcie_rreg: 111 bytes, 21 callsites
rv370_pcie_wreg: 113 bytes, 24 callsites
r600_rcu_rreg: 111 bytes, 16 callsites
r600_rcu_wreg: 113 bytes, 25 callsites
cik_didt_rreg: 106 bytes, 10 callsites
cik_didt_wreg: 107 bytes, 10 callsites
tn_smc_rreg: 106 bytes, 126 callsites
tn_smc_wreg: 107 bytes, 116 callsites
eg_cg_rreg: 107 bytes, 20 callsites
eg_cg_wreg: 108 bytes, 52 callsites

Functions r100_mm_rreg() and r100_mm_rreg() have a fast path and
a locked (slow) path. This patch deinlines only slow path.

r100_mm_rreg_slow: 78 bytes, 2083 callsites
r100_mm_wreg_slow: 81 bytes, 3570 callsites

Reduction in code size is more than 65,000 bytes:

    text     data      bss       dec     hex filename
85740176 22294680 20627456 128662312 7ab3b28 vmlinux.before
85674192 22294776 20627456 128598664 7aa4288 vmlinux
Signed-off-by: NDenys Vlasenko <dvlasenk@redhat.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This commit is the "sink CRC" version of:

commit 8c740dce
Author: Paulo Zanoni <paulo.r.zanoni@intel.com>
Date:   Fri Oct 17 18:42:03 2014 -0300
    drm/i915: disable IPS while getting the pipe CRCs.

For some unknown reason, when IPS gets enabled, the sink CRC changes.
Since hsw_enable_ips() doesn't really guarantee to enable IPS (it
depends on package C-states), we can't really predict if IPS is
enabled or disabled while running our CRC tests, so let's just
completely disable IPS while sink CRCs are being used.

If we find a way to make IPS not change the pipe CRC result, we may
want to fix IPS and then revert this patch (and 8c740dce too). While
this doesn't happen, let's merge this patch, so the IGT tests relying
on sink CRCs can work properly.

This was discovered while developing a new IGT test, which will
probably be called kms_frontbuffer_tracking.

Testcase: igt/kms_frontbuffer_tracking (not on upstream IGT yet)
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NPaulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: NRodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

It's totally broken, and since

commit d328c9d7
Author: Daniel Vetter <daniel.vetter@ffwll.ch>
Date:   Fri Apr 10 16:22:37 2015 +0200

    drm/i915: Select starting pipe bpp irrespective or the primary plane

the kernel will try to use it even for the common rgb888 framebuffers.
Ville has patches to fix it all up properly, but unfortunately they're
stuck in review limbo. And since the 4.2 feature cutoff has passed we
need to somehow  handle this regression.

Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Paulo Zanoni <przanoni@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>

chv_enable_pll() doesn't need to hold sb_lock for the entire duration of
the function. Drop the lock as soon as possible.

valleyview_set_cdclk() does a potential lock+unlock+lock+unlock cycle
with sb_lock. Grab the lock a few lines earlier so we can make do
with a single lock+unlock cycle always.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Rename dpio_lock to sb_lock to inform the reader that its primary
purpose is to protect the sideband mailbox rather than some DPIO
state.
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The primary plane frobbing was removed from the sprite code in
 commit ecce87ea
 Author: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
 Date:   Tue Apr 21 17:12:50 2015 +0300

    drm/i915: Remove implicitly disabling primary plane for now

but the intel_flush_primary_plane() calls were left behind. Replace them
with straight forward POSTING_READ() of the sprite surface address
register.

The other user of intel_flush_primary_plane() is g4x_disable_trickle_feed()
where we can just inline the steps directly.

This allows intel_flush_primary_plane() to be killed off.

Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Expecting CHV power wells to be just an extended versions of the VLV
power wells, a bunch of commented out power wells were added in
anticipation when Punit folks would implement it all. Turns out they
never did, and instead CHV has fewer power wells than VLV. Rip out all
the #if 0'ed junk that's not needed.

v2: Rename the "pipe-a" well to "display" to match VLV
    Clarify the pipe A power well relationship to pipes B and C (Deepak)
Reviewed-by: NDeepak S <deepak.s@linux.intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Not sure which LDO programming sequence delay should be used for the CHV
PHY, but the spec says that 600ns is "Used by default for initial
bringup", and the BIOS seems to use that, so let's do the same.
Reviewed-by: NDeepak S <deepak.s@linux.intel.com>
Signed-off-by: NVille Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

The value was much too low, which could cause the userspace visible
vblank counter to move backwards when the hardware counter wrapped
around.
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It is theoretically possible that a swapped out BO gets the
same GTT address, but different backing pages while being swapped in.

Instead just use another VA state to note updated areas.
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

- make it static
- fix mask/bool handling for last param
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

commit 53292cdb ("drm/i915: Workaround
to avoid lite restore with HEAD==TAIL") added a check for req0 != null
which is unnecessary.

The only way req0 could be null is if the list was empty, and this is
already addressed at the beginning of execlists_context_unqueue().
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NMichel Thierry <michel.thierry@intel.com>
Reviewed-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

dev->max_vblank_count contains the largest value that can be represented
by the hardware counter. When the hardware counter wraps around, we have
to add that value + 1 to get the same value as if the hardware counter
didn't wrap around.
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

In commit 1854d5ca
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date:   Tue Apr 7 16:20:32 2015 +0100

    drm/i915: Deminish contribution of wait-boosting from clients

we removed an atomic timer based check for allowing waitboosting and
moved it below the mutex taken during RPS. However, that mutex can be
held for long periods of time on Vallyview/Cherryview as communication
with the PCU is slow. As clients may frequently wait for results (e.g.
such as tranform feedback) we introduced contention between the client
and the RPS worker. We can take advantage of the RPS worker, by
switching the wait boost decision to use spin locks and defer the
actual reclocking to the worker.

Fixes a regression of up to 45% on Baytrail and Baswell!

v2 (Daniel):
- Use max_freq_softlimit instead of the not-yet-merged boost
  frequency.
- Don't inject a fake irq into the boost work, instead treat
  client_boost as just another legit waker.

v3: Drop the now unused mask (Chris).

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=90112
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> (v1)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>

Since we are messing with state in the worker.

v2: drop the changes in the mst worker
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Use the generic mechanism to declare a bitmap instead of unsigned long.
Signed-off-by: NJoe Perches <joe@perches.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Some of the vce clocks are automatic, others need to
be manually enabled.  For ease, just disable cg when
vce is active.

v2: rebased, call vce_v1_0_enable_mgcg directly
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Initial support for VCE 1.0 using newest firmware.

v2: rebased
v3: fix for TN
v4: fix FW size calculation
Signed-off-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This implements the function to set the vce clocks
on TN hardware.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>