MMU will need access to these.

v2. Apply fix from Rhys Kidd to send correct FECS method for STOP_CTXSW.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Future changes will want to add some additional things here, keep them
grouped together.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

This is already done during golden context creation.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Makes the code somewhat less magic.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Inserted wait-for-gr-idle in the places it seems that RM does it, seems
to prevent some random mmio timeouts on Quadro GV100.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

These were missed the first time around due to the driver version I traced
using the older registers still.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There are differences on GM200 and newer too, but we can't fix them there
as they come from firmware packages.

A request has been made to NVIDIA to release updated firmware.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There's a number of places that require this data, so let's separate out
the calculations to ensure they remain consistent.

This is incorrect for GM200 and newer, but will produce the same results
as we did before.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There's also a couple of hardcoded tables for a couple of very specific
configurations that NVGPU's algorithm didn't work for.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Required to support Volta.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

The algorithm for GM200 and newer matches RM for all the boards I have, but
I don't have enough data to try and figure something out for earlier boards,
so these will still write zeroes to the table as we did before.

The code in NVGPU isn't helpful here, it appears to handle specific cases.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

I haven't yet been able to find a fully programatic way of calculating the
same mapping as NVIDIA for GF100-GF119, so the algorithm partially depends
on data tables for specific configurations.

I couldn't find traces for every possibility, so the algorithm will switch
to a mapping similar to what GK104-GM10x use if it encounters one.  We did
the wrong thing before anyway, so shouldn't matter too much.

The algorithm used in the GK104 implementation was ported from NVGPU.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Deliberately removed from non-GP100, as RM doesn't touch it.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Pulled some init out of main per-GPC/TPC loops to match RM.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

The value for GF100 has changed here, but it matches RM now.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>