A channel may still be processed by the PBDMA even after removal, unless
it is properly kicked. Some chips are more sensible to this than others,
with GM20B triggering the issue very easily (the PBDMA will try to fetch
methods from the previously-removed channel after a new one is added).

Make sure this cannot happen by kicking the channel right after it is
disabled, and before the new runlist is submitted.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

When using the DMA-API for instmem, we may obtain a write-combined
mapping. For such cases, add a write barrier in
gk20a_instobj_release_dma() to make sure that all writes have reached
memory at this time.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Without this buffer inconsistencies may appear between the CPU
and GPU when using a PCI GPU on an ARM64 board.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Registration of the hwmon device will fail on non-PCI systems since
dev->pdev is NULL in that case. Use the more generic drm_device::dev
member that points to the same and is always set no matter the platform.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

On non-PCI devices, nobody should really care if the device does not
provide HDMI...
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

The DMA API has different semantics on different architectures.
Currently on arm64, it can only provide memory from a small pool which
dries up quickly if we attempt to allocate big buffers from it.

Do not consider that option when running on non-x86, since regular TTM
buffers are the (current) best-fit for ARM platforms.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

LTC operations timeout was set to 2ms, which may be too low for devices
that run at very low clocks (e.g. GM20B) and trigger timeout messages.

Set the timeout to the default 2s. Also remove the redundant error
messages since nvkm_wait_msec() will already display a warning.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Bits 28:29 of RUNLIST_BASE specify the memory target of the runlist. Set
it to 0x3 (SYS_MEM_NONCOHERENT) if the runlist object resides in system
memory.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Bits 28:29 of RUNLIST_BASE specify the memory target of the runlist. Set
it to 0x3 (SYS_MEM_NONCOHERENT) if the runlist object resides in system
memory.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Fix the channel id bit mask in FIFO schedule timeout error handling.

FIFO_ENGINE_STATUS_NEXT_ID is bit 27:16 thus 0x0fff0000.
FIFO_ENGINE_STATUS_ID      is bit 11:0  thus 0x00000fff.
Signed-off-by: NXia Yang <xiay@nvidia.com>
Reviewed-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

DMA mask is typically set in nouveau_ttm_init(), but this function is
called late during initialization and GK20A's instmem will have called
DMA functions before this happens.

Having a wrongly set DMA mask can result in the use of unneeded bounce
buffers. Set it early to avoid this.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NKarol Herbst <nouveau@karolherbst.de>
Reviewed-by: NMartin Peres <martin.peres@free.fr>

Signed-off-by: NKarol Herbst <nouveau@karolherbst.de>
Reviewed-by: NMartin Peres <martin.peres@free.fr>

v2: expose only if the sensor reading is valid
Signed-off-by: NKarol Herbst <nouveau@karolherbst.de>
Reviewed-by: NMartin Peres <martin.peres@free.fr>

based on Martins initial work

v3: fix ina2x9 calculations
v4: don't kmalloc(0), fix the lsb/pga stuff
v5: add a field to tell if the power reading may be invalid
    add nkvm_iccsense_read_all function
    check for the device on the i2c bus
Signed-off-by: NKarol Herbst <nouveau@karolherbst.de>
Reviewed-by: NMartin Peres <martin.peres@free.fr>

Karol Herbst:
v4: don't kmalloc(0)
v5: stricter validation
Signed-off-by: NKarol Herbst <nouveau@karolherbst.de>
Reviewed-by: NMartin Peres <martin.peres@free.fr>

Signed-off-by: NKarol Herbst <nouveau@karolherbst.de>
Reviewed-by: NMartin Peres <martin.peres@free.fr>

Add secure boot support for the GM20B chip found in Tegra X1. Secure
boot on Tegra works slightly differently from desktop, notably in the
way the WPR region is set up.

In addition, the firmware bootloaders use a slightly different header
format.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Add secure-boot for the dGPU set of GM20X chips, using the PMU as the
high-secure falcon.

This work is based on Deepak Goyal's initial port of Secure Boot to
Nouveau.

v2. use proper memory target function
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Secure falcons' firmware is managed by secboot. Do not load it in GR for
them.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Start securely-managed falcons using secboot functions since the process
for them is different from just writing CPUCTL.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

On GM200 and later GPUs, firmware for some essential falcons (notably
GR ones) must be authenticated by a NVIDIA-produced signature and
loaded by a high-secure falcon in order to be able to access privileged
registers, in a process known as Secure Boot.

Secure Boot requires building a binary blob containing the firmwares
and signatures of the falcons to be loaded. This blob is then given to
a high-secure falcon running a signed loader firmware that copies the
blob into a write-protected region, checks that the signatures are
valid, and finally loads the verified firmware into the managed falcons
and switches them to privileged mode.

This patch adds infrastructure code to support this process on chips
that require it.

v2:
- The IRQ mask of the PMU falcon was left - replace it with the proper
  irq_mask variable.
- The falcon reset procedure expecting a falcon in an initialized state,
  which was accidentally provided by the PMU subdev. Make sure that
  secboot can manage the falcon on its own.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Load firmware and bundles in GM200's constructor. The previously called
GF100 function did not care about the bundles.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

There functions are going to be used by other chips that rely on
NVIDIA-provided firmware. Export them.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Make these functions easier to use by handling memory management from
within.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

The firmwares required by GR may vary from chip to chip, especially with
the introduction of secure boot and NVIDIA-provided firmwares. Move the
firmware loading outside of gf100_gr_ctor so other chips may still call
it while managing their firmwares themselves.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Some members of gf100_gr were freed by the gk20a driver. That's not
where it should be done - free them in gf100 so other chips that use
NVIDIA-provided firmware free these structures properly.

This also removes the need for a GK20A-specific destructor.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Add memcpy functions to copy a buffer to a gpuobj and vice-versa. This
will be used by the secure boot code.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Most of the per-chipset differences will go away when we fully switch
to using the register lists provided by the firmware files, which will
leave all the remaining code "belonging" to GM200.

This is a preemptive rename from GM204 to GM200.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Upon encountering an unknown condition code, the script interpreter
is supposed to skip 'size' bytes and continue at the next devinit
token.
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

It is not advisable to perform devinit if it has already been done.
VBIOS will very likely have invoked devinit if the GPU is the primary
graphics device, but there is no accurate way to detect this fact yet.

This patch adds such a method for gf100 and later chips, by means of the
NV_PTOP_SCRATCH1_DEVINIT_COMPLETED bit. This bit is set to 1 by devinit,
and reset to 0 when the GPU is powered.
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>

We never use any nv50-specific member in this nv50_devinit_preinit().
Signed-off-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NBen Skeggs <bskeggs@redhat.com>