mainline inclusion
from mainline-v5.19
commit cf4ac3fe
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit cf4ac3fe dmaengine: idxd: fix lockdep warning on device driver removal.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Jacob reported that with lockdep debug turned on, idxd_device_driver
removal causes kernel splat from lock assert warning for
idxd_device_wqs_clear_state(). Make sure
idxd_device_wqs_clear_state() holds the wq lock for each wq when
cleaning the wq state. Move the call outside of the device spinlock.
Reported-by: NJacob Pan <jacob.jun.pan@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165231364426.986304.9294302800482492780.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.19
commit 42a1b738
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 42a1b738 dmaengine: idxd: Separate user and kernel pasid enabling.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The idxd driver always gated the pasid enabling under a single knob and
this assumption is incorrect. The pasid used for kernel operation can be
independently toggled and has no dependency on the user pasid (and vice
versa). Split the two so they are independent "enabled" flags.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165231431746.986466.5666862038354800551.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.19
commit 63c14ae6
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 63c14ae6 dmaengine: idxd: refactor wq driver enable/disable operations.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Move the core driver operations from wq driver to the drv_enable_wq() and
drv_disable_wq() functions. The move should reduce the wq driver's
knowledge of the core driver operations and prevent code confusion for
future wq drivers.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/165047301643.3841827.11222723219862233060.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.19
commit 23084545
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 23084545 dmaengine: idxd: set max_xfer and max_batch for RO device.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Load the max_xfer_size and max_batch_size values from the values read from
registers to the shadow variables. This will allow the read-only device to
display the correct values for the sysfs attributes.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164971507673.2201761.11244446608988838897.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.19
commit 3157dd0a
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 3157dd0a dmaengine: idxd: don't load pasid config until needed.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The driver currently programs the system pasid to the WQ preemptively when
system pasid is enabled. Given that a dwq will reprogram the pasid and
possibly a different pasid, the programming is not necessary. The pasid_en
bit can be set for swq as it does not need pasid programming but
needs the pasid_en bit. Remove system pasid programming on device config
write. Add pasid programming for kernel wq type on wq driver enable. The
char dev driver already reprograms the dwq on ->open() call so there's no
change.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164935607115.1660372.6734518676950372366.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.18
commit 1cd8e751
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 1cd8e751 dmaengine: idxd: skip clearing device context when device is read-only.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

If the device shows up as read-only configuration, skip the clearing of the
state as the context must be preserved for device re-enable after being
disabled.

Fixes: 0dcfe41e ("dmanegine: idxd: cleanup all device related bits after disabling device")
Reported-by: NTony Zhu <tony.zhu@intel.com>
Tested-by: NTony Zhu <tony.zhu@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164971479479.2200566.13980022473526292759.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.18
commit 12e45e89
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 12e45e89 dmaengine: idxd: fix device cleanup on disable.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

There are certain parts of WQ that needs to be cleaned up even after WQ is
disabled during the device disable. Those are the unchangeable parts for a
WQ when the device is still enabled. Move the cleanup outside of WQ state
check. Remove idxd_wq_disable_cleanup() inside idxd_wq_device_reset_cleanup()
since only the unchangeable parts need to be cleared.

Fixes: 0f225705 ("dmaengine: idxd: fix wq settings post wq disable")
Reported-by: NTony Zhu <tony.zhu@intel.com>
Tested-by: NTony Zhu <tony.zhu@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164919561905.1455025.13542366389944678346.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.18
commit ea7c8f59
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit ea7c8f59 dmaengine: idxd: restore traffic class defaults after wq reset.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

When clearing the group configurations, the driver fails to restore the
default setting for DSA 1.x based devices. Add defaults in
idxd_groups_clear_state() for traffic class configuration.

Fixes: ade8a86b ("dmaengine: idxd: Set defaults for GRPCFG traffic class")
Reported-by: NBinuraj Ravindran <binuraj.ravindran@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/164304123369.824298.6952463420266592087.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.17
commit 7ed6f1b8
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 7ed6f1b8 dmaengine: idxd: change bandwidth token to read buffers.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

DSA spec v1.2 has changed the term of "bandwidth tokens" to "read buffers"
in order to make the concept clearer. Deprecate bandwidth token
naming in the driver and convert to read buffers in order to match with
the spec and reduce confusion when reading the spec.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163951338932.2988321.6162640806935567317.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.17
commit 0f225705
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 0f225705 dmaengine: idxd: fix wq settings post wq disable.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

By the spec, wq size and group association is not changeable unless device
is disabled. Exclude clearing the shadow copy on wq disable/reset. This
allows wq type to be changed after disable to be re-enabled.

Move the size and group association to its own cleanup and only call it
during device disable.

Fixes: 0dcfe41e ("dmanegine: idxd: cleanup all device related bits after disabling device")
Reported-by: NLucas Van <lucas.van@intel.com>
Tested-by: NLucas Van <lucas.van@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163951291732.2987775.13576571320501115257.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.17
commit 403a2e23
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 403a2e23 dmaengine: idxd: change MSIX allocation based on per wq activation.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Change the driver where WQ interrupt is requested only when wq is being
enabled. This new scheme set things up so that request_threaded_irq() is
only called when a kernel wq type is being enabled. This also sets up for
future interrupt request where different interrupt handler such as wq
occupancy interrupt can be setup instead of the wq completion interrupt.

Not calling request_irq() until the WQ actually needs an irq also prevents
wasting of CPU irq vectors on x86 systems, which is a limited resource.

idxd_flush_pending_descs() is moved to device.c since descriptor flushing
is now part of wq disable rather than shutdown().
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163942149487.2412839.6691222855803875848.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.17
commit ec0d6423
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit ec0d6423 dmaengine: idxd: embed irq_entry in idxd_wq struct.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

With irq_entry already being associated with the wq in a 1:1 relationship,
embed the irq_entry in the idxd_wq struct and remove back pointers for
idxe_wq and idxd_device. In the process of this work, clean up the interrupt
handle assignment so that there's no decision to be made during submit
call on where interrupt handle value comes from. Set the interrupt handle
during irq request initialization time.

irq_entry 0 is designated as special and is tied to the device itself.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163942148362.2412839.12055447853311267866.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.17
commit 7930d855
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 7930d855 dmaengine: idxd: add knob for enqcmds retries.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Add a sysfs knob to allow tuning of retries for the kernel ENQCMDS
descriptor submission. While on host, it is not as likely that ENQCMDS
return busy during normal operations due to the driver controlling the
number of descriptors allocated for submission. However, when the driver is
operating as a guest driver, the chance of retry goes up significantly due
to sharing a wq with multiple VMs. A default value is provided with the
system admin being able to tune the value on a per WQ basis.
Suggested-by: NSanjay Kumar <sanjay.k.kumar@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163820629464.2702134.7577370098568297574.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.17
commit 92452a72
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 92452a72 dmaengine: idxd: set defaults for wq configs.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Add default values for wq size, max_xfer_size and max_batch_size. These
values should provide a general guidance for the wq configuration when
the user does not specify any specific values.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163528473483.3926048.7950067926287180976.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.17
commit 56fc39f5
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 56fc39f5 dmaengine: idxd: handle interrupt handle revoked event.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

"Interrupt handle revoked" is an event that happens when the driver is
running on a guest kernel and the VM is migrated to a new machine.
The device will trigger an interrupt that signals to the guest driver
that the interrupt handles need to be replaced.

The misc irq thread function calls a helper function to handle the
event. The function uses the WQ percpu_ref to quiesce the kernel
submissions. It then replaces the interrupt handles by requesting
interrupt handle command for each I/O MSIX vector. Once the handle is
updated, the driver will unblock the submission path to allow new
submissions.

The submitter will attempt to acquire a percpu_ref before submission. When
the request fails, it will wait on the wq_resurrect 'completion'.

The driver does anticipate the possibility of descriptors being submitted
before the WQ percpu_ref is killed. If a descriptor has already been
submitted, it will return with incorrect interrupt handle status. The
descriptor will be re-submitted with the new interrupt handle on the
completion path. For descriptors with incorrect interrupt handles,
completion interrupt won't be triggered.

At the completion of the interrupt handle refresh, the handling function
will call idxd_int_handle_refresh_drain() to issue drain descriptors to
each of the wq with associated interrupt handle. The drain descriptor will have
interrupt request set but without completion record. This will ensure all
descriptors with incorrect interrupt completion handle get drained and
a completion interrupt is triggered for the guest driver to process them.
Reviewed-by: NKevin Tian <kevin.tian@intel.com>
Co-Developed-by: NSanjay Kumar <sanjay.k.kumar@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163528420189.3925689.18212568593220415551.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.17
commit bd5970a0
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit bd5970a0 dmaengine: idxd: create locked version of idxd_quiesce() call.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Add a locked version of idxd_quiesce() call so that the quiesce can be
called with a lock in situations where the lock is not held by the caller.

In the driver probe/remove path, the lock is already held, so the raw
version can be called w/o locking.
Reviewed-by: NKevin Tian <kevin.tian@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163528418980.3925689.5841907054957931211.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.17
commit 8b67426e
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 8b67426e dmaengine: idxd: int handle management refactoring.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Attach int_handle to irq_entry. This removes the separate management of int
handles and reduces the confusion of interating through int handles that is
off by 1 count.
Reviewed-by: NKevin Tian <kevin.tian@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163528417065.3925689.11505755433684476288.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit 2efe58cf
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 2efe58cf dmaengine: idxd: cleanup completion record allocation.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

According to core-api/dma-api-howto.rst, the address from
dma_alloc_coherent is gauranteed to align to the smallest PAGE_SIZE order.
That supercedes the 64B/32B alignment requirement of the completion record.
Remove alignment adjustment code.
Tested-by: NJacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163517396063.3484297.7494385225280705372.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit e530a9f3
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit e530a9f3 dmaengine: idxd: reconfig device after device reset command.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Device reset clears the MSIXPERM table and the device registers. Re-program
the MSIXPERM table and re-enable the error interrupts post reset.

Fixes: 745e92a6 ("dmaengine: idxd: idxd: move remove() bits for idxd 'struct device' to device.c")
Reported-by: NSanjay Kumar <sanjay.k.kumar@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163054188513.2853562.12077053294595278181.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit 15af8408
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 15af8408 dmaengine: idxd: remove kernel wq type set when load configuration.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Remove setting of wq type on guest kernel during configuration load on RO
device config. The user will set the kernel wq type and this setting based
on config is not necessary.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163474724511.2607444.1876715711451990426.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit 79c4c3db
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 79c4c3db dmaengine: idxd: check GENCAP config support for gencfg register.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

DSA spec 1.2 has moved the GENCFG register under the GENCAP configuration
support with respect to writability. Add check in driver before writing to
GENCFG register.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163406171896.1303830.11217958011385656998.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit 85f604af
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 85f604af dmaengine: idxd: move out percpu_ref_exit() to ensure it's outside submission.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

percpu_ref_tryget_live() is safe to call as long as ref is between init and
exit according to the function comment. Move percpu_ref_exit() so it is
called after the dma channel is no longer valid to ensure this holds true.

Fixes: 93a40a6d ("dmaengine: idxd: add percpu_ref to descriptor submission path")
Suggested-by: NKevin Tian <kevin.tian@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163294293832.914350.10326422026738506152.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit cf84a4b9
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit cf84a4b9 dmaengine: idxd: remove interrupt disable for dev_lock.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The spinlock is not being used in hard interrupt context. There is no need
to disable irq when acquiring the lock. The interrupt thread handler also
is not in bottom half context, therefore we can also remove disabling of
the bh. Convert all dev_lock acquisition to plain spin_lock() calls.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162984026772.1939166.11504067782824765879.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit f9f4082d
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit f9f4082d dmaengine: idxd: remove interrupt disable for cmd_lock.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The cmd_lock spinlock is not being used in hard interrupt context. There is
no need to disable irq when acquiring the lock. Convert all cmd_lock
acquisition to plain spin_lock() calls.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162984027930.1939209.15758413737332339204.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit d8071323
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit d8071323 dmaengine: idxd: fix setting up priv mode for dwq.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

DSA spec says WQ priv bit is 0 if the Privileged Mode Enable field of the
PCI Express PASID capability is 0 and pasid is enabled. Make sure that the
WQCFG priv field is set correctly according to usage type. Reject config if
setting up kernel WQ type and no support. Also add the correct priv setup
for a descriptor.

Fixes: 484f910e ("dmaengine: idxd: fix wq config registers offset programming")
Cc: Ramesh Thomas <ramesh.thomas@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162939084657.903168.14160019185148244596.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 9806eb5c
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 9806eb5c dmaengine: idxd: set descriptor allocation size to threshold for swq.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Since submission is sent to limited portal, the actual wq size for shared
wq is set by the threshold rather than the wq size. When the wq type is
shared, set the allocated descriptors to the threshold.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162827151733.3459223.3829837172226042408.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit bd2f4ae5
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit bd2f4ae5 dmaengine: idxd: clear block on fault flag when clear wq.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The block on fault flag is not cleared when we disable or reset wq. This
causes it to remain set if the user does not clear it on the next
configuration load. Add clear of flag in dxd_wq_disable_cleanup()
routine.

Fixes: da32b28c ("dmaengine: idxd: cleanup workqueue config after disabling")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162803023553.3086015.8158952172068868803.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 125d1037
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 125d1037 dmanegine: idxd: add software command status.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Enabling device and wq returns standard errno and that does not provide
enough details to indicate what exactly failed. The hardware command status
is only 8bits. Expand the command status to 32bits and use the upper 16
bits to define software errors to provide more details on the exact
failure. Bit 31 will be used to indicate the error is software set as the
driver is using some of the spec defined hardware error as well.

Cc: Ramesh Thomas <ramesh.thomas@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162681373579.1968485.5891788397526827892.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit a9c17152
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit a9c17152 dmaengine: idxd: rotate portal address for better performance.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The device submission portal is on a 4k page and any of those 64bit aligned
address on the page can be used for descriptor submission. By rotating the
offset through the 4k range and prevent successive writes to the same MMIO
address, performance improvement is observed through testing.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162681372446.1968485.10634280461681015569.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 6e7f3ee9
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/i596wo
cve: na

Intel-SIG: commit 6e7f3ee9 dmaengine: idxd: move dsa_drv support to compatible mode.
incremental backporting patches for dsa/iaa on intel xeon platform.

--------------------------------

The original architecture of /sys/bus/dsa invented a scheme whereby
a single entry in the list of bus drivers, /sys/bus/drivers/dsa,
handled all device types and internally routed them to different
different drivers. Those internal drivers were invisible to
userspace.

With the idxd driver transitioned to a proper bus device-driver model,
the legacy behavior needs to be preserved due to it being exposed to
user space via sysfs. Create a compat driver to provide the legacy
behavior for /sys/bus/dsa/drivers/dsa. This should satisfy user
tool accel-config v3.2 or ealier where this behavior is expected.
If the distro has a newer accel-config then the legacy mode does
not need to be enabled.

When the compat driver binds the device (i.e. dsa0) to the dsa driver,
it will be bound to the new idxd_drv. The wq device (i.e. wq0.0) will
be bound to either the dmaengine_drv or the user_drv. The dsa_drv
becomes a routing mechansim for the new drivers. It will not support
additional external drivers that are implemented later.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637468705.744545.4399080971745974435.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 448c3de8
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 448c3de8 dmaengine: idxd: create user driver for wq 'device'.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The original architecture of /sys/bus/dsa invented a scheme whereby a
single entry in the list of bus drivers, /sys/bus/drivers/dsa, handled
all device types and internally routed them to different drivers.
Those internal drivers were invisible to userspace. Now, as
/sys/bus/dsa wants to grow support for alternate drivers for a given
device, for example vfio-mdev instead of kernel-internal-dmaengine, a
proper bus device-driver model is needed. The first step in that process
is separating the existing omnibus/implicit "dsa" driver into proper
individual drivers registered on /sys/bus/dsa. Establish the
idxd_user_drv driver that controls the enabling and disabling of the
wq and also register and unregister a char device to allow user space
to mmap the descriptor submission portal.

The cdev related bits are moved to the cdev driver probe/remove and out of
the drv_enabe/disable_wq() calls. These bits are exclusive to the cdev
operation and not part of the generic enable/disable of the wq device.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637467578.744545.10203997610072341376.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 0cda4f69
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 0cda4f69 dmaengine: idxd: create dmaengine driver for wq 'device'.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The original architecture of /sys/bus/dsa invented a scheme whereby a
single entry in the list of bus drivers, /sys/bus/drivers/dsa, handled
all device types and internally routed them to different drivers.
Those internal drivers were invisible to userspace. Now, as
/sys/bus/dsa wants to grow support for alternate drivers for a given
device, for example vfio-mdev instead of kernel-internal-dmaengine, a
proper bus device-driver model is needed. The first step in that process
is separating the existing omnibus/implicit "dsa" driver into proper
individual drivers registered on /sys/bus/dsa. Establish the
idxd_dmaengine_drv driver that controls the enabling and disabling of the
wq and also register and unregister the dma channel.

idxd_wq_alloc_resources() and idxd_wq_free_resources() also get moved to
the dmaengine driver. The resources (dma descriptors allocation and setup)
are only used by the dmaengine driver and should only happen when it loads.

The char dev driver (cdev) related bits are left in the __drv_enable_wq()
and __drv_disable_wq() calls to be moved when we split out the char dev
driver just like how the dmaengine driver is split out.

WQ autoload support is not expected currently. With the amount of
configuration needed for the device, the wq is always expected to
be enabled by a tool (or via sysfs) rather than auto enabled at driver
load.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637467033.744545.12330636655625405394.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 034b3290
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 034b3290 dmaengine: idxd: create idxd_device sub-driver.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The original architecture of /sys/bus/dsa invented a scheme whereby a
single entry in the list of bus drivers, /sys/bus/drivers/dsa, handled
all device types and internally routed them to different drivers.
Those internal drivers were invisible to userspace. Now, as
/sys/bus/dsa wants to grow support for alternate drivers for a given
device, for example vfio-mdev instead of kernel-internal-dmaengine, a
proper bus device-driver model is needed. The first step in that process
is separating the existing omnibus/implicit "dsa" driver into proper
individual drivers registered on /sys/bus/dsa. Establish the idxd_drv
driver that control the enabling and disabling of the accelerator device.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637466439.744545.15210886092627144577.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 745e92a6
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 745e92a6 dmaengine: idxd: idxd: move remove() bits for idxd 'struct device' to device.c.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Move the code related to a ->remove() function for the idxd
'struct device' to device.c to prep for the idxd device
sub-driver in device.c.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637464768.744545.15797285510999151668.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit bd42805b
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit bd42805b dmaengine: idxd: move probe() bits for idxd 'struct device' to device.c.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Move the code related to a ->probe() function for the idxd
'struct device' to device.c to prep for the idxd device
sub-driver in device.c.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637464189.744545.17423830646786162194.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 69e4f8be
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 69e4f8be dmaengine: idxd: move wq_disable() to device.c.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Move the wq_disable() function to device.c in preparation of setting up the
idxd internal sub-driver framework. No logic changes.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637461775.744545.9644048686618957886.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 1f2bb403
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 1f2bb403 dmaengine: idxd: move wq_enable() to device.c.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Move the wq_enable() function to device.c in preparation of setting up the
idxd internal sub-driver framework. No logic changes.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637461176.744545.3806109011554118998.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit f52058ae
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit f52058ae dmaengine: idxd: remove IDXD_DEV_CONF_READY.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The IDXD_DEV_CONF_READY state flag is no longer needed. The current
implementation uses this flag to stop the device from doing
configuration until the pci driver probe has completed. With the
driver architecture going towards multiple sub-driver attached to
the dsa_bus, this is no longer feasible. The sub-drivers will be
allowed to probe and return with failure when they are not ready
to complete the probe rather than using a state flag to gate the
probing.

There is no expectation that the devices auto-attach to a driver.
Userspace configuration is expected to setup the device before
enabling.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637460633.744545.8902095097471365420.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 53499d1f
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 53499d1f dmaengine: idxd: have command status always set.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The cached command status is only set when the write back status is
is passed in. Move the variable set outside of the check so it is
always set.

Fixes: 0d5c10b4 ("dmaengine: idxd: add work queue drain support")
Reported-by: NRamesh Thomas <ramesh.thomas@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162274329740.1822314.3443875665504707588.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 0dcfe41e
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 0dcfe41e dmanegine: idxd: cleanup all device related bits after disabling device.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The previous state cleanup patch only performed wq state cleanups. This
does not go far enough as when device is disabled or reset, the state
for groups and engines must also be cleaned up. Add additional state
cleanup beyond wq cleanup. Tie those cleanups directly to device
disable and reset, and wq disable and reset.

Fixes: da32b28c ("dmaengine: idxd: cleanup workqueue config after disabling")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162285154108.2096632.5572805472362321307.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>