The power management code calls into the display code for
certain things.  If certain power management sysfs attributes
are called before the driver has finished initializing all of
the hardware we can run into problems with uninitialized
modesetting state.  Add a check to make sure modesetting
init has completed to the bandwidth update callbacks to
fix this.  Can be triggered by the tlp and laptop start
up scripts depending on the timing.

bugs:
https://bugzilla.kernel.org/show_bug.cgi?id=83611
https://bugs.freedesktop.org/show_bug.cgi?id=85771Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

It isn't necessary for command streams generated by the kernel (at least
not while we aren't storing ring or indirect buffers in VRAM).
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Since 3.16-rc1 we have this new failure:

When the userspace XOrg ddx schedules vblank events to
trigger deferred kms-pageflips, e.g., via the OML_sync_control
extension call glXSwapBuffersMscOML(), or if a glXSwapBuffers()
is called immediately after completion of a previous swapbuffers
call, e.g., in a tight rendering loop with minimal rendering,
it happens frequently that the pageflip ioctl() is executed
within the same vblank in which a previous kms-pageflip completed,
or - for deferred swaps - always one vblank earlier than requested
by the client app.

This causes premature pageflips and detection of failure by
the ddx, e.g., XOrg log warnings like...

"(WW) RADEON(1): radeon_dri2_flip_event_handler: Pageflip
completion event has impossible msc 201025 < target_msc 201026"

... and error/invalid return values of glXWaitForSbcOML() and
Intel_swap_events extension.

Reason is the new way in which kms-pageflips are programmed
since 3.16.

This commit changes the time window in which the hw can
execute pending programmed pageflips. Before, a pending flip
would get executed anywhere within the vblank interval. Now
a pending flip only gets executed at the leading edge of
vblank (start of front porch), making sure that a invocation
of the pageflip ioctl() within a given vblank interval will
only lead to pageflip completion in the following vblank.

Tested to death on a DCE-4 card.
Signed-off-by: NMario Kleiner <mario.kleiner.de@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Moving the pm resume up in the init order to fix
dpm seems to have regressed somes cases with the old
pm code.  Move it back to late resume.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

We need more control over the ordering of dpm init with
respect to the rest of the asic.  Specifically, the SMC
has to be initialized before the rlc and cg/pg.  The pm
code currently initializes late in the driver, but we need
it to happen much earlier so move pm handling into the asic
specific callbacks.

This makes dpm more reliable and makes clockgating work
properly on CIK parts and should help on SI parts as well.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Spotted by Brad Smith when porting to OpenBSD.
Noticed-by: NBrad Smith <brad@comstyle.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

This adds spinlocks to protect access to other
indirect register apertures.  These indirect spaces are
used pretty infrequently and we haven't had an reported
problems, but better safe than sorry.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Calculate the low and high watermarks based on the low and high
clocks for the current power state.  The dynamic pm hw will select
the appropriate watermark based on the internal dpm state.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

The current radeon driver initialization routines, when using KMS, are written
so that the IRQ installation routine is called before initializing the WB buffer
and the CP rings. With some ASICs, though, the IRQ routine tries to access the
GFX_INDEX ring causing a call to RREG32 with the value of -1 in
radeon_fence_read. This, in turn causes the system to completely hang with some
cards, requiring a hard reset.

A call stack that can cause such a hang looks like this (using rv515 ASIC for the
example here):
 * rv515_init (rv515.c)
 * radeon_irq_kms_init (radeon_irq_kms.c)
 * drm_irq_install (drm_irq.c)
 * radeon_driver_irq_preinstall_kms (radeon_irq_kms.c)
 * rs600_irq_process (rs600.c)
 * radeon_fence_process - due to SW interrupt (radeon_fence.c)
 * radeon_fence_read (radeon_fence.c)
 * hang due to RREG32(-1)

The patch moves the IRQ installation to the card startup routine, after the ring
has been initialized, but before the IRQ has been set. This fixes the issue, but
requires a check to see if the IRQ is already installed, as is the case in the
system resume codepath.
I have tested the patch on three machines using the rv515, the rv770 and the
evergreen ASIC. They worked without issues.

This seems to be a known issue and has been reported on several bug tracking
sites by various distributions (see links below). Most of reports recommend
booting the system with KMS disabled and then enabling KMS by reloading the
radeon module. For some reason, this was indeed a usable workaround, however,
UMS is now deprecated and disabled by default.

Bug reports:
https://bugzilla.redhat.com/show_bug.cgi?id=845745
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/561789
https://bbs.archlinux.org/viewtopic.php?id=156964Signed-off-by: NAdis Hamzić <adis@hamzadis.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Just disabling the mem requests should be enough, but
that doesn't seem to work correctly on efi systems.

v2: blank displays first, then disable.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Need to wait for the new addresses to take affect before
re-enabling the MC.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Doesn't affect anything as the same address gets written
in both cases.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org

Some chips seem to need a little delay after blacking out
the MC before the requests actually stop. Stop DMAR errors
reported by Shuah Khan.
Reported-by: NShuah Khan <shuahkhan@gmail.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Along the same lines of what was done for evergreen+
in the last kernel.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

Convert #include "..." to #include <path/...> in drivers/gpu/.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
Acked-by: NDave Airlie <airlied@redhat.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: NDave Jones <davej@redhat.com>

Let's allow GCC to optimize better.

This exposed some five unused functions, but this patch doesn't remove them.
Signed-off-by: NLauri Kasanen <cand@gmx.com>
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>

It seems we can not update the crtc scanout address. After disabling
crtc, update to base address do not take effect after crtc being
reenable leading to at least frame being scanout from the old crtc
base address. Disabling crtc display request lead to same behavior.

So after changing the vram address if we don't keep crtc disabled
we will have the GPU trying to read some random system memory address
with some iommu this will broke the crtc engine and will lead to
broken display and iommu error message.

So to avoid this, disable crtc. For flicker less boot we will need
to avoid moving the vram start address.

This patch should also fix :

https://bugs.freedesktop.org/show_bug.cgi?id=42373

Cc: <stable@vger.kernel.org>
Signed-off-by: NJerome Glisse <jglisse@redhat.com>

Making it easier to control when it is executed.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

The IB pool is in gart memory, so it is completely
superfluous to unpin / repin it on suspend / resume.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

Removing all the different error messages and
having just one standard behaviour over all
chipset generations.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Each ring type may need a different variant.
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König<christian.koenig@amd.com>
Reviewed-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

If accel is not working many subsystem such as the ib pool might not be
initialized properly that can lead to segfault inside kernel when cs
ioctl is call with non working acceleration. To avoid this make sure
the accel working flag is false when an error in GPU startup happen and
return EBUSY from cs ioctl if accel is not working.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Correct spelling "reseting" to "resetting" in
drivers/gpu/drm/radeon/rv515.c
Signed-off-by: NMasanari Iida <standby24x7@gmail.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

Somewhat specializaed sub-allocator designed to perform sub-allocation
for command buffer not only for current cs ioctl but for future command
submission ioctl as well. Patch also convert current ib pool to use
the sub allocator. Idea is that ib poll buffer can be share with other
command buffer submission not having 64K granularity.

v2 Harmonize pool handling and add suspend/resume callback to pin/unpin
sa bo (tested on rv280, rv370, r420, rv515, rv610, rv710, redwood, cayman,
rs480, rs690, rs880)
v3 Simplify allocator
v4 Fix radeon_ib_get error path to properly free fence
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Add a start fence driver helper function which will be call
once for each ring and will compute cpu/gpu addr for fence
depending on wether to use wb buffer or scratch reg.

This patch replace initialize fence driver separately which
was broken in regard of GPU lockup. The fence list for created,
emited, signaled must be initialize once and only from the
asic init callback not from the startup call back which is
call from the gpu reset.

v2: With this in place we no longer need to know the number of
    rings in fence_driver_init, also writing to the scratch reg
    before knowing its offset is a bad idea.

v3: rebase on top of change to previous patch in the serie
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

That naming seems to make more sense, since we not
only want to run PM4 rings with it.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Replace cp, cp1 and cp2 members with just an array
of radeon_cp structs.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Give all asic and radeon_ring_* functions a
radeon_cp parameter, so they know the ring to work with.
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

For supporting multiple CP ring buffers, async DMA
engines and UVD.  We still need a way to synchronize
between engines.

v2 initialize unused fence driver ring to avoid issue in
   suspend/unload
Signed-off-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NChristian König <deathsimple@vodafone.de>
Reviewed-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NPaul Bolle <pebolle@tiscali.nl>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@gmail.com>

When writeback is enabled, the GPU shadows writes to certain
registers into a buffer in memory.  The driver can then read
the values from the shadow rather than reading back from the
register across the bus.  Writeback can be disabled by setting
the no_wb module param to 1.

On r6xx/r7xx/evergreen, the following registers are shadowed:
- CP scratch registers
- CP read pointer
- IH write pointer
On r1xx-rr5xx, the following registers are shadowed:
- CP scratch registers
- CP read pointer

v2:
- Combine wb patches for r6xx-evergreen and r1xx-r5xx
- Writeback is disabled on AGP boards since it tends to be
unreliable on AGP using the gart.
- Check radeon_wb_init return values properly.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Logic was:
if (mode0 && mode1)
else if (mode0)
else

Should be:
if (mode0 && mode1)
else if (mode0)
else if (mode1)

Otherwise we may end up calculating the priority regs with
unitialized values.

Fixes:
https://bugzilla.kernel.org/show_bug.cgi?id=16492Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Cc: stable@kernel.org
Signed-off-by: NDave Airlie <airlied@redhat.com>

On systems using kexec, the new kernel is booted straight from the old kernel, without any warning to the graphics driver. So the GPU is basically left as-is in a running state, however the CPU side is completly reset.

Without stating the saneness of anyone using kexec on live systems, we should at least try not to crash the GPU. This patch resets 3 registers to 0 that could cause bad things to happen to the running system.

This allows kexec to work on a Power6/RN50 system.
Signed-off-by: NDave Airlie <airlied@redhat.com>

The asics in question have the following requirements with regard to
their gart setups:

1. The GART aperture size has to be in the form of 2^X bytes, where X is from 25 to 31
2. The GART aperture MC base has to be aligned to a boundary equal to the size of the
aperture.
3. The GART page table has to be aligned to the boundary equal to the size of the table.
4. The GART page table size is: table_entry_size * (aperture_size / page_size)
5. The GART page table has to be allocated in non-paged, non-cached, contiguous system
memory.

This patch takes care 2.  The rest should already be handled properly.

This fixes a regression noticed by: Torsten Kaiser <just.for.lkml@googlemail.com>
Tested-by: NTorsten Kaiser <just.for.lkml@googlemail.com>
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

- Separate dynpm and profile based power management methods.  You can select the pm method
  by echoing the selected method ("dynpm" or "profile") to power_method in sysfs.
- Expose basic 4 profile in profile method
  "default" - default clocks
  "auto" - select between low and high based on ac/dc state
  "low" - DC, low power mode
  "high" - AC, performance mode
  The current base profile is "default", but it should switched to "auto" once we've tested
  on more systems.  Switching the state is a matter of echoing the requested profile to
  power_profile in sysfs.  The lowest power states are selected automatically when dpms turns
  the monitors off in all states but default.
- Remove dynamic fence-based reclocking for the moment.  We can revisit this later once we
  have basic pm in.
- Move pm init/fini to modesetting path.  pm is tightly coupled with display state.  Make sure
  display side is initialized before pm.
- Add pm suspend/resume functions to make sure pm state is properly reinitialized on resume.
- Remove dynpm module option.  It's now selectable via sysfs.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Will be used by nouveau driver also in the near future.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This simplify and improve GPU reset for R1XX-R6XX hw, it's
not 100% reliable here are result:
- R1XX/R2XX works bunch of time in a row, sometimes it
  seems it can work indifinitly
- R3XX/R3XX the most unreliable one, sometimes you will be
  able to reset few times, sometimes not even once
- R5XX more reliable than previous hw, seems to work most
  of the times but once in a while it fails for no obvious
  reasons (same status than previous reset just no same
  happy ending)
- R6XX/R7XX are lot more reliable with this patch, still
  it seems that it can fail after a bunch (reset every
  2sec for 3hour bring down the GPU & computer)

This have been tested on various hw, for some odd reasons
i wasn't able to lockup RS480/RS690 (while they use to
love locking up).

Note that on R1XX-R5XX the cursor will disapear after
lockup haven't checked why, switch to console and back
to X will restore cursor.

Next step is to record the bogus command that leaded to
the lockup.

V2 Fix r6xx resume path to avoid reinitializing blit
module, use the gpu_lockup boolean to avoid entering
inifinite waiting loop on fence while reiniting the GPU
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Patch rename gpu_reset to asic_reset in prevision of having
gpu_reset doing more stuff than just basic asic reset.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

- Add module option to force the display priority
  0 = auto, 1 = normal, 2 = high
- Default to high on r3xx/r4xx/rv515 chips
  Fixes flickering problems during heavy acceleration
  due to underflow to the display controllers
- Fill in minimal support for RS600

v2 - update display priority when bandwidth is updated
so the user can change the parameter at runtime and it
will take affect on the next modeset.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>