When writeback is enabled, the GPU shadows writes to certain
registers into a buffer in memory.  The driver can then read
the values from the shadow rather than reading back from the
register across the bus.  Writeback can be disabled by setting
the no_wb module param to 1.

On r6xx/r7xx/evergreen, the following registers are shadowed:
- CP scratch registers
- CP read pointer
- IH write pointer
On r1xx-rr5xx, the following registers are shadowed:
- CP scratch registers
- CP read pointer

v2:
- Combine wb patches for r6xx-evergreen and r1xx-r5xx
- Writeback is disabled on AGP boards since it tends to be
unreliable on AGP using the gart.
- Check radeon_wb_init return values properly.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Move common code to init function.

v2: make sure the bo is pinned after init as well.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This code was originally for forcing some clocks on certain asics.
However, this code was later moved to asic specific functions
for all of the affected asics.  The only users of the original
code at this point were r600, rv770, and evergreen and the code
was not relevant for those asics.  So, remove it.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

commit: 812d0469
drm/radeon/kms/r7xx: add workaround for hw issue with HDP flush
breaks on AGP boards since there is no VRAM gart table.

This patch fixes the issue by creating a VRAM scratch page so that
can be used on both AGP and PCIE.

Fixes:
https://bugs.freedesktop.org/show_bug.cgi?id=29834Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Cc: stable@kernel.org
Signed-off-by: NDave Airlie <airlied@redhat.com>

Use of HDP_*_COHERENCY_FLUSH_CNTL can cause a hang in certain
situations.  Add workaround.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Cc: stable@kernel.org
Signed-off-by: NDave Airlie <airlied@redhat.com>

Userspace needs this information to access tiled
buffers via the CPU.

v2: rebased on evergreen accel changes
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

rv6xx/rv7xx/evergreen families supported; older asics did
not have an internal thermal sensor.

Note, not all oems use the internal thermal sensor, so it's
only exposed in cases where it is used.

Note also, that most laptops use an oem specific ACPI solution for
GPU thermal information rather than using the internal thermal
sensor directly.

v2: export millidegrees celsius, use hwmon device properly.
v3: fix Kconfig
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

HDP non surface should cover the whole VRAM but we were misscomputing
the size and we endup in some case not covering the VRAM at all (if
VRAM size were > 1G). Covering more than the VRAM size shouldn't be
an issue.

Fix : https://bugs.freedesktop.org/show_bug.cgi?id=28016

[airlied: add evergreen fix]
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

agd5f: rebased
Signed-off-by: NRafał Miłecki <zajec5@gmail.com>
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This fixes FDO bug #28375, it's kind of regression, so quite important to have
it for .35.

V2: Fix on RV770+ as well. All other chipsets have only one clock mode per
state.

V3: I'm out of luck today. Grepped for voltage in r*.c and missed evergreen.

agd5f: rebased
Signed-off-by: NRafał Miłecki <zajec5@gmail.com>
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

track the current voltage level and avoid setting it
if the requested voltage is already set.

v2: check voltage type before checking current voltage
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

I missed these in the r6xx commit.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Remove the drm_resource wrappers and directly use the
actual PCI and/or platform functions in their place.

[airlied: fixup nouveau properly to build]
Signed-off-by: NJordan Crouse <jcrouse@codeaurora.org>
Reviewed-by: NMatt Turner <mattst88@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

- Separate dynpm and profile based power management methods.  You can select the pm method
  by echoing the selected method ("dynpm" or "profile") to power_method in sysfs.
- Expose basic 4 profile in profile method
  "default" - default clocks
  "auto" - select between low and high based on ac/dc state
  "low" - DC, low power mode
  "high" - AC, performance mode
  The current base profile is "default", but it should switched to "auto" once we've tested
  on more systems.  Switching the state is a matter of echoing the requested profile to
  power_profile in sysfs.  The lowest power states are selected automatically when dpms turns
  the monitors off in all states but default.
- Remove dynamic fence-based reclocking for the moment.  We can revisit this later once we
  have basic pm in.
- Move pm init/fini to modesetting path.  pm is tightly coupled with display state.  Make sure
  display side is initialized before pm.
- Add pm suspend/resume functions to make sure pm state is properly reinitialized on resume.
- Remove dynpm module option.  It's now selectable via sysfs.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

- pm_misc() - handles voltage, pcie lanes, and other non
clock related power mode settings.  Currently disabled.
Needs further debugging

- pm_prepare() - disables crtc mem requests right now.
All memory clients need to be disabled when changing
memory clocks.  This function can be expanded to include
disabling fb access as well.

- pm_finish() - enable active memory clients.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This patch enable the use of unmappable VRAM thanks to
previous TTM infrastructure change.

V2 update after io_mem_reserve/io_mem_free callback balancing
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

The command processor (CP) fetches command buffers and
feeds the GPU.  This patch requires the evergreen
family me and pfp ucode files.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This patch cleanup the fence code, it drops the timeout field of
fence as the time to complete each IB is unpredictable and shouldn't
be bound.

The fence cleanup lead to GPU lockup detection improvement, this
patch introduce a callback, allowing to do asic specific test for
lockup detection. In this patch the CP is use as a first indicator
of GPU lockup. If CP doesn't make progress during 1second we assume
we are facing a GPU lockup.

To avoid overhead of testing GPU lockup frequently due to fence
taking time to be signaled we query the lockup callback every
500msec. There is plenty code comment explaining the design & choise
inside the code.

This have been tested mostly on R3XX/R5XX hw, in normal running
destkop (compiz firefox, quake3 running) the lockup callback wasn't
call once (1 hour session). Also tested with forcing GPU lockup and
lockup was reported after the 1s CP activity timeout.

V2 switch to 500ms timeout so GPU lockup get call at least 2 times
   in less than 2sec.
V3 store last jiffies in fence struct so on ERESTART, EBUSY we keep
   track of how long we already wait for a given fence
V4 make sure we got up to date cp read pointer so we don't have
   false positive
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

radeon_gart_fini might call GART unbind callback function which
might try to access GART table but if gart_disable is call first
the GART table will be unmapped so any access to it will oops.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

- rs780/880 were using the wrong bandwidth functions
- convert r1xx-r4xx to use the same pm sclk/mclk structs as
r5xx+
- move bandwidth setup to a common function
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This fixes some issues with the last gfx init patch.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Look up i2c bus in the power table and expose it.
You'll need to load a hwmon driver for any chips
on the bus, this patch just exposes the bus.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h

percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files.  percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed.  Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability.  As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

  http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
  only the necessary includes are there.  ie. if only gfp is used,
  gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
  blocks and try to put the new include such that its order conforms
  to its surrounding.  It's put in the include block which contains
  core kernel includes, in the same order that the rest are ordered -
  alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
  doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
  because the file doesn't have fitting include block), it prints out
  an error message indicating which .h file needs to be added to the
  file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
   over 4000 files, deleting around 700 includes and adding ~480 gfp.h
   and ~3000 slab.h inclusions.  The script emitted errors for ~400
   files.

2. Each error was manually checked.  Some didn't need the inclusion,
   some needed manual addition while adding it to implementation .h or
   embedding .c file was more appropriate for others.  This step added
   inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
   from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
   e.g. lib/decompress_*.c used malloc/free() wrappers around slab
   APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
   editing them as sprinkling gfp.h and slab.h inclusions around .h
   files could easily lead to inclusion dependency hell.  Most gfp.h
   inclusion directives were ignored as stuff from gfp.h was usually
   wildly available and often used in preprocessor macros.  Each
   slab.h inclusion directive was examined and added manually as
   necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
   were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
   distributed build env didn't work with gcov compiles) and a few
   more options had to be turned off depending on archs to make things
   build (like ipr on powerpc/64 which failed due to missing writeq).

   * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
   * powerpc and powerpc64 SMP allmodconfig
   * sparc and sparc64 SMP allmodconfig
   * ia64 SMP allmodconfig
   * s390 SMP allmodconfig
   * alpha SMP allmodconfig
   * um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
   a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: NTejun Heo <tj@kernel.org>
Guess-its-ok-by: NChristoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

In essence this creates a home for all asic specific declarations in
radeon_asic.h
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Signed-off-by: NRafał Miłecki <zajec5@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This patch properly set visible VRAM and enforce any pinned buffer
to be into visible VRAM. We might later add a flag to release this
constraint for some newer hw more clever than previous.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

- updated swizzle modes for backend map setup
- fix programming of a few gfx regs
- properly handle pipe/backend setup on LE cards
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This patch fixes occlusion queries and rendering errors
on rv740 boards. Hardcoding the backend map is not an optimal
solution, but a better fix is being worked on.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This patch fixes occlusion queries and rendering errors
on rv740 boards. Hardcoding the backend map is not an optimal
solution, but a better fix is being worked on.
Signed-off-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

Get rid of _location and use _start/_end also simplify the
computation of vram_start|end & gtt_start|end. For R1XX-R2XX
we place VRAM at the same address of PCI aperture, those GPU
shouldn't have much memory and seems to behave better when
setup that way. For R3XX and newer we place VRAM at 0. For
R6XX-R7XX AGP we place VRAM before or after AGP aperture this
might limit to limit the VRAM size but it's very unlikely.
For IGP we don't change the VRAM placement.

Tested on (compiz,quake3,suspend/resume):
PCI/PCIE:RV280,R420,RV515,RV570,RV610,RV710
AGP:RV100,RV280,R420,RV350,RV620(RPB*),RV730
IGP:RS480(RPB*),RS690,RS780(RPB*),RS880

RPB: resume previously broken

V2 correct commit message to reflect more accurately the bug
and move VRAM placement to 0 for most of the GPU to avoid
limiting VRAM.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

this uses a new entrypoint to invalidate gart entries instead of using 0.
Changed to rather than pointing to 0 address point empty entry to dummy
page. This might help to avoid hard lockup if for some wrong
reasons GPU try to access unmapped GART entry.

I'm not 100% sure this is going to work, we probably need to allocate
a dummy page and point all the GTT entries at it similiar to what AGP does.
but we can test this first I suppose.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

This patch add cs checker to r600/r700 hw. Command stream checking
will rewrite some of the cs value in order to restrict GPU access
to BO size. This doesn't break old userspace but just enforce safe
value. It should break any things that was using the r600/r700 cs
ioctl to do forbidden things (malicious software), though we are
not aware of such things.

Here is the list of thing we check :
- enforcing resource size
- enforcing color buffer slice tile max, will restrict cb access
- enforcing db buffer slice tile max, will restrict db access

We don't check for shader bigger than the BO in which they are
supposed to be, such use would lead to GPU lockup and is harmless
from security POV, as far as we can tell (note that even checking
for this wouldn't prevent someone to write bogus shader that lead
to lockup).

This patch has received as much testing as humanly possible with
old userspace to check that it didn't break such configuration.
However not all the applications out there were tested, thus it
might broke some odd, rare applications.

[airlied: fix rules for cs checker for parallel builds]
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

In suspend path we unmap the GART table while in cleaning up
path we will unbind buffer and thus try to write to unmapped
GART leading to oops. In order to avoid this we don't call the
suspend path in cleanup path. Cleanup path is clever enough
to desactive GPU like the suspend path is doing, thus this was
redondant.

Tested on: RV370, R420, RV515, RV570, RV610, RV770 (all PCIE)
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

VGA might be overwritting VRAM and corrupt our blit shader leading
to corruption, it likely won't happen if you load fbcon right after
radeon. Thanks to Shawn Starr and Andre Maasikas for tracking down
this issue.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

If an error happen in r600_blit_prepare_copy report it rather
than WARNING and keeping execution. For instance if ib allocation
failed we did just warn about but then latter tried to access
NULL ib ptr causing oops. This patch also protect r600_copy_blit
with a mutex as otherwise one process might overwrite blit temporary
data with new one possibly leading to GPU lockup.

Should partialy or totaly fix:
https://bugzilla.redhat.com/show_bug.cgi?id=553279

V2 failing blit initialization is not fatal, fallback to memcpy when
this happen
V3 init blit before startup as we pin in startup, remove duplicate
code (this one was actualy tested unlike V2)
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

If ib initialization failed don't try to test ib as it will result
in an oops (accessing NULL ib buffer ptr).
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@linux.ie>

To avoid hw doing anythings after we disabled PCIE GART, fully
disable IRQ at suspend. Also cleanup a bit the ih structure
and process function.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Reviewed-by: NAlex Deucher <alexdeucher@gmail.com>
Signed-off-by: NDave Airlie <airlied@linux.ie>

Use same common function to disable agp so we replace the GART
callback by the proper one when we do so. This fix oops if
radeon_agp_init report failure.

This patch also move radeon_agp_init out of *_mc_init for r600
& rv770 so that we can have a similar behavior than for previous
hw, ie if agp_init fails it will fallback to GPU GART and disable
AGP.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

r600 blit cleanup path need to check if a bo was allocated before
trying to free or unpin it. This patch add this check and avoid
oops when the initialization on r6xx or r7xx hw fails.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>

In some case we weren't releasing the AGP device at module unloading.
This leaded to unfunctional AGP at next module load. This patch make
sure we release the AGP bus if we acquire it.
Signed-off-by: NJerome Glisse <jglisse@redhat.com>
Signed-off-by: NDave Airlie <airlied@redhat.com>