Rename cxl_afu_reset() to __cxl_afu_reset() to we can reuse this function name
in the API.
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Acked-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

Rework __detach_context() and cxl_context_detach() so we can reuse them in the
kernel API.
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Acked-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

Add cookie parameter to afu_release_irqs() so that we can pass in a different
cookie than the context structure.  This will be useful for other kernel
drivers that want to call this but get their own cookie back in the interrupt
handler.

Update all existing call sites.
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Acked-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

The afu fd release path was identified as a significant bottleneck in
the overall performance of cxl. While an optimal AFU design would
minimise the need to close & reopen the AFU fd, it is not always
practical to avoid.

The bottleneck seems to be down to the call to synchronize_rcu(), which
will block until every other thread is guaranteed to be out of an RCU
critical section. Replace it with call_rcu() to free the context
structures later so we can return to the application sooner.

This reduces the time spent in the fd release path from 13356 usec to
13.3 usec - about a 100x speed up.
Reported-by: NFei K Chen <uchen@cn.ibm.com>
Signed-off-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

Export the "AFU Error Buffer" via sysfs attribute (afu_err_buf). AFU
error buffer is used by the AFU to report application specific
errors. The contents of this buffer are AFU specific and are intended to
be interpreted by the application interacting with the afu.
Suggested-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NVaibhav Jain <vaibhav@linux.vnet.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

Given a file descriptor on an afu device, libcxl currently uses the
major/minor number obtained from fstat on the fd to construct path to
the afu's sysfs directory. However it is possible that rather than using
one of the device in /dev/cxl, a kernel driver creates its own device
which export generic cxl interface to the userspace. This causes
problems with libcxl as it tries to use a wrong major/minor number to
construct the sysfs path and fail.

So this patch introduces a new ioctl called CXL_IOCTL_GET_AFU_ID on the
afu file descriptor to fetch the cxl_afu_id struct that holds the
card/offset-id and mode information. These info is then used by libcxl to
construct the correct path to the afu sysfs directory.

Testing:
	- Build against pseries be/le configs
	- Testing with corresponding libcxl changes to verify that it constructs
	  right sysfs path to the afu.
Signed-off-by: NVaibhav Jain <vaibhav@linux.vnet.ibm.com>
Acked-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

An AFU may optionally contain one or more PCIe like configuration
records, which can be used to identify the AFU.

This patch adds support for exposing the raw config space and the
vendor, device and class code under sysfs. These will appear in a
subdirectory of the AFU device corresponding with the configuration
record number, e.g.

cat /sys/class/cxl/afu0.0/cr0/vendor
0x1014

cat /sys/class/cxl/afu0.0/cr0/device
0x4350

cat /sys/class/cxl/afu0.0/cr0/class
0x120000

hexdump -C /sys/class/cxl/afu0.0/cr0/config
00000000  14 10 50 43 00 00 00 00  06 00 00 12 00 00 00 00  |..PC............|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000100

These files behave in much the same way as the equivalent files for PCI
devices, with one exception being that the config file is currently
read-only and restricted to the root user. It is not necessarily
required to be this strict, but we currently do not have a compelling
use-case to make it writable and/or world-readable, so I erred on the
side of being restrictive.
Signed-off-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

Adds reset to sysfs which will PERST the card. If load_image_on_perst is set
to "user" or "factory", the PERST will cause that image to be loaded.

load_image_on_perst is set to "user" for production.

"none" could be used for debugging. The PSL trace arrays are preserved which
then can be read through debugfs.

PERST also triggers CAPP recovery. An HMI comes in, which is handled by EEH.
EEH unbinds the driver, calls into Sapphire to reinitialize the PHB, then
rebinds the driver.
Signed-off-by: NRyan Grimm <grimm@linux.vnet.ibm.com>
Acked-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

Select defaults such that a PERST causes flash image reload.  Select which
image based on what the card is set up to load.

CXL_VSEC_PERST_LOADS_IMAGE selects whether PERST assertion causes flash image
load.

CXL_VSEC_PERST_SELECT_USER selects which image is loaded on the next PERST.

cxl_update_image_control writes these bits into the VSEC.
Signed-off-by: NRyan Grimm <grimm@linux.vnet.ibm.com>
Acked-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

Upon inspection of the implementation specific registers, it was
discovered that the high bit of the implementation specific RXCTL
register was enabled, which enables the DEADB00F debug feature.

The debug feature causes MMIO reads to a disabled AFU to respond with
0xDEADB00F instead of all Fs. In general this should not be visible as
the kernel will only allow MMIO access to enabled AFUs, but there may be
some circumstances where an AFU may become disabled while it is use.
One such case would be an AFU designed to only be used in the dedicated
process mode and to disable itself after it has completed it's work
(however even in that case the effects of this debug flag would be
limited as the userspace application must have completed any required
MMIO accesses before the AFU disables itself with or without the flag).

This patch removes the debug flag and replaces the magic value
programmed into this register with a preprocessor define so it is
clearer what the rest of this initialisation does.
Signed-off-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

If we need to force detach a context (e.g. due to EEH or simply force
unbinding the driver) we should prevent the userspace contexts from
being able to access the Problem State Area MMIO region further, which
they may have mapped with mmap().

This patch unmaps any mapped MMIO regions when detaching a userspace
context.

Cc: stable@vger.kernel.org
Signed-off-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

We had a known sleep while atomic bug if a CXL device was forcefully
unbound while it was in use. This could occur as a result of EEH, or
manually induced with something like this while the device was in use:

echo 0000:01:00.0 > /sys/bus/pci/drivers/cxl-pci/unbind

The issue was that in this code path we iterated over each context and
forcefully detached it with the contexts_lock spin lock held, however
the detach also needed to take the spu_mutex, and call schedule.

This patch changes the contexts_lock to a mutex so that we are not in
atomic context while doing the detach, thereby avoiding the sleep while
atomic.

Also delete the related TODO comment, which suggested an alternate
solution which turned out to not be workable.

Cc: stable@vger.kernel.org
Signed-off-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

Currently all interrupts generated by cxl are named "cxl".  This is not very
informative as we can't distinguish between cards, AFUs, error interrupts, user
contexts and user interrupts numbers.  Being able to distinguish them is useful
for setting affinity.

This patch gives each of these names in /proc/interrupts.

A two card CAPI system, with afu0.0 having 2 active contexts each with 4 user
IRQs each, will now look like this:

    % grep cxl /proc/interrupts
    444:          0  OPAL ICS 141312 Level     cxl-card1-err
    445:          0  OPAL ICS 141313 Level     cxl-afu1.0-err
    446:          0  OPAL ICS 141314 Level     cxl-afu1.0
    462:          0  OPAL ICS 2052 Level     cxl-afu0.0-pe0-1
    463:      75517  OPAL ICS 2053 Level     cxl-afu0.0-pe0-2
    468:          0  OPAL ICS 2054 Level     cxl-afu0.0-pe0-3
    469:          0  OPAL ICS 2055 Level     cxl-afu0.0-pe0-4
    470:          0  OPAL ICS 2056 Level     cxl-afu0.0-pe1-1
    471:      75506  OPAL ICS 2057 Level     cxl-afu0.0-pe1-2
    472:          0  OPAL ICS 2058 Level     cxl-afu0.0-pe1-3
    473:          0  OPAL ICS 2059 Level     cxl-afu0.0-pe1-4
    502:       1066  OPAL ICS 2050 Level     cxl-afu0.0
    514:          0  OPAL ICS 2048 Level     cxl-card0-err
    515:          0  OPAL ICS 2049 Level     cxl-afu0.0-err
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

If an AFU has a hardware bug that causes it to acknowledge a context
terminate or remove while that context has outstanding transactions, it
is possible for the kernel to receive an interrupt for that context
after we have removed it from the context list.

The kernel will not be able to demultiplex the interrupt (or worse - if
we have already reallocated the process handle we could mis-attribute it
to the new context), and printed a big scary warning.

It did not acknowledge the interrupt, which would effectively halt
further translation fault processing on the PSL.

This patch makes the warning clearer about the likely cause of the issue
(i.e. hardware bug) to make it obvious to future AFU designers of what
needs to be fixed. It also prints out the process handle which can then
be matched up with hardware and software traces for debugging.

It also acknowledges the interrupt to the PSL with either an address
error or acknowledge, so that the PSL can continue with other
translations.
Signed-off-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Currently all interrupts generated by cxl are named "cxl".  This is not very
informative as we can't distinguish between cards, AFUs, error interrupts, user
contexts and user interrupts numbers.  Being able to distinguish them is useful
for setting affinity.

This patch gives each of these names in /proc/interrupts.

A two card CAPI system, with afu0.0 having 2 active contexts each with 4 user
IRQs each, will now look like this:

    % grep cxl /proc/interrupts
    444:          0  OPAL ICS 141312 Level     cxl-card1-err
    445:          0  OPAL ICS 141313 Level     cxl-afu1.0-err
    446:          0  OPAL ICS 141314 Level     cxl-afu1.0
    462:          0  OPAL ICS 2052 Level     cxl-afu0.0-pe0-1
    463:      75517  OPAL ICS 2053 Level     cxl-afu0.0-pe0-2
    468:          0  OPAL ICS 2054 Level     cxl-afu0.0-pe0-3
    469:          0  OPAL ICS 2055 Level     cxl-afu0.0-pe0-4
    470:          0  OPAL ICS 2056 Level     cxl-afu0.0-pe1-1
    471:      75506  OPAL ICS 2057 Level     cxl-afu0.0-pe1-2
    472:          0  OPAL ICS 2058 Level     cxl-afu0.0-pe1-3
    473:          0  OPAL ICS 2059 Level     cxl-afu0.0-pe1-4
    502:       1066  OPAL ICS 2050 Level     cxl-afu0.0
    514:          0  OPAL ICS 2048 Level     cxl-card0-err
    515:          0  OPAL ICS 2049 Level     cxl-afu0.0-err
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

If an AFU has a hardware bug that causes it to acknowledge a context
terminate or remove while that context has outstanding transactions, it
is possible for the kernel to receive an interrupt for that context
after we have removed it from the context list.

The kernel will not be able to demultiplex the interrupt (or worse - if
we have already reallocated the process handle we could mis-attribute it
to the new context), and printed a big scary warning.

It did not acknowledge the interrupt, which would effectively halt
further translation fault processing on the PSL.

This patch makes the warning clearer about the likely cause of the issue
(i.e. hardware bug) to make it obvious to future AFU designers of what
needs to be fixed. It also prints out the process handle which can then
be matched up with hardware and software traces for debugging.

It also acknowledges the interrupt to the PSL with either an address
error or acknowledge, so that the PSL can continue with other
translations.
Signed-off-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>

This is the core of the cxl driver.

It adds support for using cxl cards in the powernv environment only (ie POWER8
bare metal). It allows access to cxl accelerators by userspace using the
/dev/cxl/afuM.N char devices.

The kernel driver has no knowledge of the function implemented by the
accelerator. It provides services to userspace via the /dev/cxl/afuM.N
devices. When a program opens this device and runs the start work IOCTL, the
accelerator will have coherent access to that processes memory using the same
virtual addresses. That process may mmap the device to access any MMIO space
the accelerator provides.  Also, reads on the device will allow interrupts to
be received. These services are further documented in a later patch in
Documentation/powerpc/cxl.txt.

Documentation of the cxl hardware architecture and userspace API is provided in
subsequent patches.
Signed-off-by: NIan Munsie <imunsie@au1.ibm.com>
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NMichael Ellerman <mpe@ellerman.id.au>