Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Inserted wait-for-gr-idle in the places it seems that RM does it, seems
to prevent some random mmio timeouts on Quadro GV100.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

These were missed the first time around due to the driver version I traced
using the older registers still.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There are differences on GM200 and newer too, but we can't fix them there
as they come from firmware packages.

A request has been made to NVIDIA to release updated firmware.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes it easier to diff against RM traces.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There's a number of places that require this data, so let's separate out
the calculations to ensure they remain consistent.

This is incorrect for GM200 and newer, but will produce the same results
as we did before.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There's also a couple of hardcoded tables for a couple of very specific
configurations that NVGPU's algorithm didn't work for.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Required to support Volta.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

RM and NVGPU both have a variant of this, we probably should too.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

We weren't placing higher TPC IDs in the right place on some configurations.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Removed from GK110[B]/GK208 as RM traces show it not being touched.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

The algorithm for GM200 and newer matches RM for all the boards I have, but
I don't have enough data to try and figure something out for earlier boards,
so these will still write zeroes to the table as we did before.

The code in NVGPU isn't helpful here, it appears to handle specific cases.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

I don't think this is done after Fermi, NVGPU used to do it but removed
the code, and I've not seen RM traces touching it either.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

I haven't yet been able to find a fully programatic way of calculating the
same mapping as NVIDIA for GF100-GF119, so the algorithm partially depends
on data tables for specific configurations.

I couldn't find traces for every possibility, so the algorithm will switch
to a mapping similar to what GK104-GM10x use if it encounters one.  We did
the wrong thing before anyway, so shouldn't matter too much.

The algorithm used in the GK104 implementation was ported from NVGPU.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Also fixes some GPUs where we write too many registers.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

GM20B now also shares the same code, as NVGPU shows it doesn't need
special treatment.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>