mainline inclusion
from mainline-v5.15
commit 3a5cc016
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 3a5cc016 dmaengine: idxd: remove bus shutdown.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Remove ->shutdown() function for the dsa bus as it does not do anything and
is not necessary.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637462319.744545.10383189484257042066.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 69e4f8be
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 69e4f8be dmaengine: idxd: move wq_disable() to device.c.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Move the wq_disable() function to device.c in preparation of setting up the
idxd internal sub-driver framework. No logic changes.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637461775.744545.9644048686618957886.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 1f2bb403
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 1f2bb403 dmaengine: idxd: move wq_enable() to device.c.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Move the wq_enable() function to device.c in preparation of setting up the
idxd internal sub-driver framework. No logic changes.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637461176.744545.3806109011554118998.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit f52058ae
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit f52058ae dmaengine: idxd: remove IDXD_DEV_CONF_READY.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The IDXD_DEV_CONF_READY state flag is no longer needed. The current
implementation uses this flag to stop the device from doing
configuration until the pci driver probe has completed. With the
driver architecture going towards multiple sub-driver attached to
the dsa_bus, this is no longer feasible. The sub-drivers will be
allowed to probe and return with failure when they are not ready
to complete the probe rather than using a state flag to gate the
probing.

There is no expectation that the devices auto-attach to a driver.
Userspace configuration is expected to setup the device before
enabling.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637460633.744545.8902095097471365420.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 700af3a0
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 700af3a0 dmaengine: idxd: add 'struct idxd_dev' as wrapper for conf_dev.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Add a 'struct idxd_dev' that wraps the 'struct device' for idxd conf_dev
that registers with the dsa bus. This is introduced in order to deal with
multiple different types of 'devices' that are registered on the dsa_bus
when the compat driver needs to route them to the correct driver to attach.
The bind() call now can determine the type of device and then do the
appropriate driver matching.

Reviewed-by Dan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637460065.744545.584492831446090984.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit da5a11d7
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit da5a11d7 dmaengine: idxd: add driver name.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Add name field in idxd_device_driver so we don't have to touch the
'struct device_driver' during declaration.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637459517.744545.7572915135318813722.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 3ecfc913
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 3ecfc913 dmaengine: idxd: add driver register helper.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Add helper functions for dsa-driver registration similar to other
bus-types. In particular, do not require dsa-drivers to open-code the
bus, owner, and mod_name fields. Let registration and unregistration
operate on the 'struct idxd_device_driver' instead of the raw /
embedded 'struct device_driver'.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637458949.744545.14996726325385482050.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 53499d1f
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 53499d1f dmaengine: idxd: have command status always set.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The cached command status is only set when the write back status is
is passed in. Move the variable set outside of the check so it is
always set.

Fixes: 0d5c10b4 ("dmaengine: idxd: add work queue drain support")
Reported-by: NRamesh Thomas <ramesh.thomas@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162274329740.1822314.3443875665504707588.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit e753a64b
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit e753a64b dmaengine: idxd: Add wq occupancy information to sysfs attribute.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Add occupancy information to wq sysfs attribute. Attribute will show
wq occupancy data if "WQ Occupancy Support" field in WQCAP is 1. It
displays the number of entries currently in this WQ. This is provided
as an estimate and should not be relied on to determine whether there
is space in the WQ. The data is to provide information to user apps
for flow control.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162275745546.1857062.8765615879420582018.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 0dcfe41e
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 0dcfe41e dmanegine: idxd: cleanup all device related bits after disabling device.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The previous state cleanup patch only performed wq state cleanups. This
does not go far enough as when device is disabled or reset, the state
for groups and engines must also be cleaned up. Add additional state
cleanup beyond wq cleanup. Tie those cleanups directly to device
disable and reset, and wq disable and reset.

Fixes: da32b28c ("dmaengine: idxd: cleanup workqueue config after disabling")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162285154108.2096632.5572805472362321307.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 53b50458
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 53b50458 dmaengine: idxd: Simplify code and axe the use of a deprecated API.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The wrappers in include/linux/pci-dma-compat.h should go away.

Replace 'pci_set_dma_mask/pci_set_consistent_dma_mask' by an equivalent
and less verbose 'dma_set_mask_and_coherent()' call.

Even if the code may look different, it should have exactly the same
run-time behavior.
If pci_set_dma_mask(64) fails and pci_set_dma_mask(32) succeeds, then
pci_set_consistent_dma_mask(64) will also fail.
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Acked-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/70c8a3bc67e41c5fefb526ecd64c5174c1e2dc76.1625720835.git.christophe.jaillet@wanadoo.frSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.14
commit 6b4b87f2
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 6b4b87f2 dmaengine: idxd: fix submission race window.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

Deviation from upstream:
Merge commit 88c5d0a2 Merge branch 'fixes' into next.

--------------------------------

Konstantin observed that when descriptors are submitted, the descriptor is
added to the pending list after the submission. This creates a race window
with the slight possibility that the descriptor can complete before it
gets added to the pending list and this window would cause the completion
handler to miss processing the descriptor.

To address the issue, the addition of the descriptor to the pending list
must be done before it gets submitted to the hardware. However, submitting
to swq with ENQCMDS instruction can cause a failure with the condition of
either wq is full or wq is not "active".

With the descriptor allocation being the gate to the wq capacity, it is not
possible to hit a retry with ENQCMDS submission to the swq. The only
possible failure can happen is when wq is no longer "active" due to hw
error and therefore we are moving towards taking down the portal. Given
this is a rare condition and there's no longer concern over I/O
performance, the driver can walk the completion lists in order to retrieve
and abort the descriptor.

The error path will set the descriptor to aborted status. It will take the
work list lock to prevent further processing of worklist. It will do a
delete_all on the pending llist to retrieve all descriptors on the pending
llist. The delete_all action does not require a lock. It will walk through
the acquired llist to find the aborted descriptor while add all remaining
descriptors to the work list since it holds the lock. If it does not find
the aborted descriptor on the llist, it will walk through the work
list. And if it still does not find the descriptor, then it means the
interrupt handler has removed the desc from the llist but is pending on
the work list lock and will process it once the error path releases the
lock.

Fixes: eb15e715 ("dmaengine: idxd: add interrupt handle request and release support")
Reported-by: NKonstantin Ananyev <konstantin.ananyev@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162628855747.360485.10101925573082466530.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.14
commit 7eb25da1
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 7eb25da1 dmaengine: idxd: fix sequence for pci driver remove() and shutdown().
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

->shutdown() call should only be responsible for quiescing the device.
Currently it is doing PCI device tear down. This causes issue when things
like MMIO mapping is removed while idxd_unregister_devices() will trigger
removal of idxd device sub-driver and still initiates MMIO writes to the
device. Another issue is with the unregistering of idxd 'struct device',
the memory context gets freed. So the teardown calls are accessing freed
memory and can cause kernel oops. Move all the teardown bits that doesn't
belong in shutdown to ->remove() call. Move unregistering of the idxd
conf_dev 'struct device' to after doing all the teardown to free all
the memory that's no longer needed.

Fixes: 47c16ac2 ("dmaengine: idxd: fix idxd conf_dev 'struct device' lifetime")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162629983901.395844.17964803190905549615.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.14
commit d5c10e0f
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit d5c10e0f dmaengine: idxd: fix setup sequence for MSIXPERM table.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The MSIX permission table should be programmed BEFORE request_irq()
happens. This prevents any possibility of an interrupt happening before the
MSIX perm table is setup, however slight.

Fixes: 6df0e6c5 ("dmaengine: idxd: clear MSIX permission entry on shutdown")
Sign-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162456741222.1138073.1298447364671237896.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 6cfd9e62
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 6cfd9e62 dmaengine: idxd: assign MSIX vectors to each WQ rather than roundrobin.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

Deviation from upstream:
Merge commit da435aed dmaengine: idxd: fix array index when int_handles are being used.

--------------------------------

IOPS increased when changing MSIX vector to per WQ from roundrobin.
Allows descriptor to be completed by the submitter improves caching
locality.
Suggested-by: NKonstantin Ananyev <konstantin.ananyev@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Acked-by: NKonstantin Ananyev <konstantin.ananyev@intel.com>
Link: https://lore.kernel.org/r/162456717326.1130457.15258077196523268356.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.14
commit da435aed
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit da435aed dmaengine: idxd: fix array index when int_handles are being used.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The index to the irq vector should be local and has no relation to
the assigned interrupt handle. Assign the MSIX interrupt index that is
programmed for the descriptor. The interrupt handle only matters when it
comes to hardware descriptor programming.

Fixes: eb15e715 ("dmaengine: idxd: add interrupt handle request and release support")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162456176939.1121476.3366256009925001897.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.14
commit 8ba89a3c
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 8ba89a3c dmaengine: idxd: fix desc->vector that isn't being updated.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Missing update for desc->vector when the wq vector gets updated. This
causes the desc->vector to always be at 0.

Fixes: da435aed ("dmaengine: idxd: fix array index when int_handles are being used")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162628784374.353761.4736602409627820431.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit ac24a2dc
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit ac24a2dc dmaengine: idxd: add missing percpu ref put on failure.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

When enqcmds() fails, exit path is missing a percpu_ref_put(). This can
cause failure on shutdown path when the driver is attempting to quiesce the
wq. Add missing percpu_ref_put() call on the error exit path.

Fixes: 93a40a6d ("dmaengine: idxd: add percpu_ref to descriptor submission path")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162456170168.1121236.7240941044089212312.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.14
commit 25905f60
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 25905f60 dmaengine: idxd: Change license on idxd.h to LGPL.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

This file was given GPL-2.0 license. But LGPL-2.1 makes more sense
as it needs to be used by libraries outside of the kernel source tree.
Signed-off-by: NTony Luck <tony.luck@intel.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.14
commit 33f9f3c3
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 33f9f3c3 dmaengine: idxd: remove devm allocation for idxd->int_handles.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Allocation of idxd->int_handles was merged incorrectly for the 5.13 merge
window. The devm_kcalloc should've been regular kcalloc due to devm_*
removal series for the driver.

Fixes: eb15e715 ("dmaengine: idxd: add interrupt handle request and release support")
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162060710518.130816.11349798049329202863.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.14
commit 58cb138e
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 58cb138e dmaengine: idxd: Remove redundant variable cdev_ctx.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Variable cdev_ctx is set to '&ictx[wq->idxd->data->type]' but this
value is not used, hence it is a redundant assignment and can be
removed.

Clean up the following clang-analyzer warning:

drivers/dma/idxd/cdev.c:300:2: warning: Value stored to 'cdev_ctx' is
never read [clang-analyzer-deadcode.DeadStores].
Reported-by: NAbaci Robot <abaci@linux.alibaba.com>
Signed-off-by: NJiapeng Chong <jiapeng.chong@linux.alibaba.com>
Acked-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/1620298847-33127-1-git-send-email-jiapeng.chong@linux.alibaba.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 99b18e88
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 99b18e88 dmaengine: idxd: Fix missing error code in idxd_cdev_open().
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The error code is missing in this code scenario, add the error code
'-EINVAL' to the return value 'rc'.

Eliminate the follow smatch warning:

drivers/dma/idxd/cdev.c:113 idxd_cdev_open() warn: missing error code
'rc'.
Reported-by: NAbaci Robot <abaci@linux.alibaba.com>
Signed-off-by: NJiapeng Chong <jiapeng.chong@linux.alibaba.com>
Acked-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/1622628446-87909-1-git-send-email-jiapeng.chong@linux.alibaba.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit ddf742d4
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit ddf742d4 dmaengine: idxd: Add missing cleanup for early error out in probe call.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The probe call stack is missing some cleanup when things fail in the
middle. Add the appropriate cleanup routines to make sure we exit
gracefully.

Fixes: a39c7cd0 ("dmaengine: idxd: removal of pcim managed mmio mapping")
Reported-by: NNikhil Rao <nikhil.rao@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162197061707.392656.15760573520817310791.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 1c4841cc
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 1c4841cc dmaengine: idxd: add engine 'struct device' missing bus type assignment.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

engine 'struct device' setup is missing assigning the bus type. Add it to
dsa_bus_type.

Fixes: 75b91130 ("dmaengine: idxd: fix engine conf_dev lifetime")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161947841562.984844.17505646725993659651.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 74b2fc88
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 74b2fc88 dmaengine: idxd: Use cpu_feature_enabled().
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

When testing x86 feature bits, use cpu_feature_enabled() so that
build-disabled features can remain off, regardless of what CPUID says.

Fixes: 8e50d392 ("dmaengine: idxd: Add shared workqueue support")
Signed-off-by: NBorislav Petkov <bp@suse.de>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Acked-By: NVinod Koul <vkoul@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 0bde4444
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 0bde4444 dmaengine: idxd: Enable IDXD performance monitor support.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Add the code needed in the main IDXD driver to interface with the IDXD
perfmon implementation.

[ Based on work originally by Jing Lin. ]
Reviewed-by: NDave Jiang <dave.jiang@intel.com>
Reviewed-by: NKan Liang <kan.liang@linux.intel.com>
Signed-off-by: NTom Zanussi <tom.zanussi@linux.intel.com>
Link: https://lore.kernel.org/r/a5564a5583911565d31c2af9234218c5166c4b2c.1619276133.git.zanussi@kernel.orgSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 81dd4d4d
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 81dd4d4d dmaengine: idxd: Add IDXD performance monitor support.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Implement the IDXD performance monitor capability (named 'perfmon' in
the DSA (Data Streaming Accelerator) spec [1]), which supports the
collection of information about key events occurring during DSA and
IAX (Intel Analytics Accelerator) device execution, to assist in
performance tuning and debugging.

The idxd perfmon support is implemented as part of the IDXD driver and
interfaces with the Linux perf framework.  It has several features in
common with the existing uncore pmu support:

  - it does not support sampling
  - does not support per-thread counting

However it also has some unique features not present in the core and
uncore support:

  - all general-purpose counters are identical, thus no event constraints
  - operation is always system-wide

While the core perf subsystem assumes that all counters are by default
per-cpu, the uncore pmus are socket-scoped and use a cpu mask to
restrict counting to one cpu from each socket.  IDXD counters use a
similar strategy but expand the scope even further; since IDXD
counters are system-wide and can be read from any cpu, the IDXD perf
driver picks a single cpu to do the work (with cpu hotplug notifiers
to choose a different cpu if the chosen one is taken off-line).

More specifically, the perf userspace tool by default opens a counter
for each cpu for an event.  However, if it finds a cpumask file
associated with the pmu under sysfs, as is the case with the uncore
pmus, it will open counters only on the cpus specified by the cpumask.
Since perfmon only needs to open a single counter per event for a
given IDXD device, the perfmon driver will create a sysfs cpumask file
for the device and insert the first cpu of the system into it.  When a
user uses perf to open an event, perf will open a single counter on
the cpu specified by the cpu mask.  This amounts to the default
system-wide rather than per-cpu counting mentioned previously for
perfmon pmu events.  In order to keep the cpu mask up-to-date, the
driver implements cpu hotplug support for multiple devices, as IDXD
usually enumerates and registers more than one idxd device.

The perfmon driver implements basic perfmon hardware capability
discovery and configuration, and is initialized by the IDXD driver's
probe function.  During initialization, the driver retrieves the total
number of supported performance counters, the pmu ID, and the device
type from idxd device, and registers itself under the Linux perf
framework.

The perf userspace tool can be used to monitor single or multiple
events depending on the given configuration, as well as event groups,
which are also supported by the perfmon driver.  The user configures
events using the perf tool command-line interface by specifying the
event and corresponding event category, along with an optional set of
filters that can be used to restrict counting to specific work queues,
traffic classes, page and transfer sizes, and engines (See [1] for
specifics).

With the configuration specified by the user, the perf tool issues a
system call passing that information to the kernel, which uses it to
initialize the specified event(s).  The event(s) are opened and
started, and following termination of the perf command, they're
stopped.  At that point, the perfmon driver will read the latest count
for the event(s), calculate the difference between the latest counter
values and previously tracked counter values, and display the final
incremental count as the event count for the cycle.  An overflow
handler registered on the IDXD irq path is used to account for counter
overflows, which are signaled by an overflow interrupt.

Below are a couple of examples of perf usage for monitoring DSA events.

The following monitors all events in the 'engine' category.  Becuuse
no filters are specified, this captures all engine events for the
workload, which in this case is 19 iterations of the work generated by
the kernel dmatest module.

Details describing the events can be found in Appendix D of [1],
Performance Monitoring Events, but briefly they are:

  event 0x1:  total input data processed, in 32-byte units
  event 0x2:  total data written, in 32-byte units
  event 0x4:  number of work descriptors that read the source
  event 0x8:  number of work descriptors that write the destination
  event 0x10: number of work descriptors dispatched from batch descriptors
  event 0x20: number of work descriptors dispatched from work queues

 # perf stat -e dsa0/event=0x1,event_category=0x1/,
                dsa0/event=0x2,event_category=0x1/,
		dsa0/event=0x4,event_category=0x1/,
		dsa0/event=0x8,event_category=0x1/,
		dsa0/event=0x10,event_category=0x1/,
		dsa0/event=0x20,event_category=0x1/
		  modprobe dmatest channel=dma0chan0 timeout=2000
		  iterations=19 run=1 wait=1

     Performance counter stats for 'system wide':

                 5,332      dsa0/event=0x1,event_category=0x1/
                 5,327      dsa0/event=0x2,event_category=0x1/
                    19      dsa0/event=0x4,event_category=0x1/
                    19      dsa0/event=0x8,event_category=0x1/
                     0      dsa0/event=0x10,event_category=0x1/
                    19      dsa0/event=0x20,event_category=0x1/

          21.977436186 seconds time elapsed

The command below illustrates filter usage with a simple example.  It
specifies that MEM_MOVE operations should be counted for the DSA
device dsa0 (event 0x8 corresponds to the EV_MEM_MOVE event - Number
of Memory Move Descriptors, which is part of event category 0x3 -
Operations. The detailed category and event IDs are available in
Appendix D, Performance Monitoring Events, of [1]).  In addition to
the event and event category, a number of filters are also specified
(the detailed filter values are available in Chapter 6.4 (Filter
Support) of [1]), which will restrict counting to only those events
that meet all of the filter criteria.  In this case, the filters
specify that only MEM_MOVE operations that are serviced by work queue
wq0 and specifically engine number engine0 and traffic class tc0
having sizes between 0 and 4k and page size of between 0 and 1G result
in a counter hit; anything else will be filtered out and not appear in
the final count.  Note that filters are optional - any filter not
specified is assumed to be all ones and will pass anything.

 # perf stat -e dsa0/filter_wq=0x1,filter_tc=0x1,filter_sz=0x7,
                filter_eng=0x1,event=0x8,event_category=0x3/
		  modprobe dmatest channel=dma0chan0 timeout=2000
		  iterations=19 run=1 wait=1

     Performance counter stats for 'system wide':

       19      dsa0/filter_wq=0x1,filter_tc=0x1,filter_sz=0x7,
               filter_eng=0x1,event=0x8,event_category=0x3/

          21.865914091 seconds time elapsed

The output above reflects that the unspecified workload resulted in
the counting of 19 MEM_MOVE operation events that met the filter
criteria.

[1]: https://software.intel.com/content/www/us/en/develop/download/intel-data-streaming-accelerator-preliminary-architecture-specification.html

[ Based on work originally by Jing Lin. ]
Reviewed-by: NDave Jiang <dave.jiang@intel.com>
Reviewed-by: NKan Liang <kan.liang@linux.intel.com>
Signed-off-by: NTom Zanussi <tom.zanussi@linux.intel.com>
Link: https://lore.kernel.org/r/0c5080a7d541904c4ad42b848c76a1ce056ddac7.1619276133.git.zanussi@kernel.orgSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit a1610461
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit a1610461 dmaengine: idxd: remove MSIX masking for interrupt handlers.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Remove interrupt masking and just let the hard irq handler keep
firing for new events. This is less of a performance impact vs
the MMIO readback inside the pci_msi_{mask,unmas}_irq(). Especially
with a loaded system those flushes can be stuck behind large amounts
of MMIO writes to flush. When guest kernel is running on top of VFIO
mdev, mask/unmask causes a vmexit each time and is not desirable.
Suggested-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Link: https://lore.kernel.org/r/161894523436.3210025.1834640110556139277.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 53b2ee7f
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 53b2ee7f dmaengine: idxd: device cmd should use dedicated lock.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Create a dedicated lock for device command operations. Put the device
command operation under finer grained locking instead of using the
idxd->dev_lock.
Suggested-by: NSanjay Kumar <sanjay.k.kumar@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161894525685.3210132.16160045731436382560.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 5b0c68c4
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 5b0c68c4 dmaengine: idxd: support reporting of halt interrupt.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Unmask the halt error interrupt so it gets reported to the interrupt
handler. When halt state interrupt is received, quiesce the kernel
WQs and unmap the portals to stop submission.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161894441167.3202472.9485946398140619501.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit cf5f86a7
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit cf5f86a7 dmaengine: idxd: enable SVA feature for IOMMU.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Enable IOMMU_DEV_FEAT_SVA before attempt to bind pasid. This is needed
according to iommu_sva_bind_device() comment. Currently Intel IOMMU code
does this before bind call. It really needs to be controlled by the driver.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161894440621.3202472.17644507396206848134.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 8241571f
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 8241571f dmaengine: idxd: convert sprintf() to sysfs_emit() for all usages.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Convert sprintf() to sysfs_emit() in order to check buffer overrun on sysfs
outputs.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161894440044.3202472.13926639619695319753.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit eb15e715
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit eb15e715 dmaengine: idxd: add interrupt handle request and release support.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

DSA spec states that when Request Interrupt Handle and Release Interrupt
Handle command bits are set in the CMDCAP register, these device commands
must be supported by the driver.

The interrupt handle is programmed in a descriptor. When Request Interrupt
Handle is not supported, the interrupt handle is the index of the desired
entry in the MSI-X table. When the command is supported, driver must use
the command to obtain a handle to be programmed in the submitted
descriptor.

A requested handle may be revoked. After the handle is revoked, any use of
the handle will result in Invalid Interrupt Handle error.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161894439422.3202472.17579543737810265471.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 8c66bbdc
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 8c66bbdc dmaengine: idxd: add support for readonly config mode.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The read-only configuration mode is defined by the DSA spec as a mode of
the device WQ configuration. When GENCAP register bit 31 is set to 0,
the device is in RO mode and group configuration and some fields of the
workqueue configuration registers are read-only and reflect the fixed
configuration of the device. Add support for RO mode. The driver will
load the values from the registers directly setup all the internally
cached data structures based on the device configuration.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161894438847.3202472.6317563824045432727.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 93a40a6d
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 93a40a6d dmaengine: idxd: add percpu_ref to descriptor submission path.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Current submission path has no way to restrict the submitter from
stop submiting on shutdown path or wq disable path. This provides a way to
quiesce the submission path.

Modeling after 'struct reqeust_queue' usage of percpu_ref. One of the
abilities of per_cpu reference counting is the ability to stop new
references from being taken while awaiting outstanding references to be
dropped. On wq shutdown, we want to block any new submissions to the kernel
workqueue and quiesce before disabling. The percpu_ref allows us to block
any new submissions and wait for any current submission calls to finish
submitting to the workqueue.

A percpu_ref is embedded in each idxd_wq context to allow control for
individual wq. The wq->wq_active counter is elevated before calling
movdir64b() or enqcmds() to submit a descriptor to the wq and dropped once
the submission call completes. The function is gated by
percpu_ref_tryget_live(). On shutdown with percpu_ref_kill() called, any
new submission would be blocked from acquiring a ref and failed. Once all
references are dropped for the wq, shutdown can continue.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161894438293.3202472.14894701611500822232.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 435b512d
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 435b512d dmaengine: idxd: remove detection of device type.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Move all static data type for per device type to an idxd_driver_data data
structure. The data can be attached to the pci_device_id and provided by
the pci probe function. This removes a lot of unnecessary type detection
and setup code.
Suggested-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161852988924.2203940.2787590808682466398.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 4b73e4eb
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 4b73e4eb dmaengine: idxd: iax bus removal.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

There is no need to have an additional bus for the IAX device. The removal
of IAX will change user ABI as /sys/bus/iax will no longer exist.
The iax device will be moved to the dsa bus. The device id for dsa and
iax will now be combined rather than unique for each device type in order
to accommodate the iax devices. This is in preparation for fixing the
sub-driver code for idxd. There's no hardware deployment for Sapphire
Rapids platform yet, which means that users have no reason to have
developed scripts against this ABI. There is some exposure to
released versions of accel-config, but those are being fixed up and
an accel-config upgrade is reasonable to get IAX support. As far as
accel-config is concerned IAX support starts when these devices appear
under /sys/bus/dsa, and old accel-config just assumes that an empty /
missing /sys/bus/iax just means a lack of platform support.

Fixes: f25b4638 ("dmaengine: idxd: add IAX configuration support in the IDXD driver")
Suggested-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161852988298.2203940.4529909758034944428.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit defe49f9
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit defe49f9 dmaengine: idxd: fix group conf_dev lifetime.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Remove devm_* allocation and fix group->conf_dev 'struct device'
lifetime. Address issues flagged by CONFIG_DEBUG_KOBJECT_RELEASE.
Add release functions in order to free the allocated memory at the
group->conf_dev destruction time.
Reported-by: NJason Gunthorpe <jgg@nvidia.com>
Fixes: bfe1d560 ("dmaengine: idxd: Init and probe for Intel data accelerators")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161852987144.2203940.8830315575880047.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 75b91130
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 75b91130 dmaengine: idxd: fix engine conf_dev lifetime.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Remove devm_* allocation and fix engine->conf_dev 'struct device'
lifetime. Address issues flagged by CONFIG_DEBUG_KOBJECT_RELEASE.
Add release functions in order to free the allocated memory at the
engine conf_dev destruction time.
Reported-by: NJason Gunthorpe <jgg@nvidia.com>
Fixes: bfe1d560 ("dmaengine: idxd: Init and probe for Intel data accelerators")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161852986460.2203940.16603218225412118431.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.13
commit 7c5dd23e
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 7c5dd23e dmaengine: idxd: fix wq conf_dev 'struct device' lifetime.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Remove devm_* allocation and fix wq->conf_dev 'struct device' lifetime.
Address issues flagged by CONFIG_DEBUG_KOBJECT_RELEASE. Add release
functions in order to free the allocated memory for the wq context at
device destruction time.
Reported-by: NJason Gunthorpe <jgg@nvidia.com>
Fixes: bfe1d560 ("dmaengine: idxd: Init and probe for Intel data accelerators")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/161852985907.2203940.6840120734115043753.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>