mainline inclusion
from mainline-v5.17
commit 5d78abb6
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 5d78abb6 dmaengine: idxd: rework descriptor free path on failure.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

Deviation from upstream:
Merge commit 5cb664fb Merge branch 'fixes' into next.

--------------------------------

Refactor the completion function to allow skipping of descriptor freeing on
the submission failure path. This completely removes descriptor freeing
from the submit failure path and leave the responsibility to the caller.
Reviewed-by: NKevin Tian <kevin.tian@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163528416222.3925689.12859769271667814762.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit 8affd8a4
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 8affd8a4 dmaengine: idxd: fix missed completion on abort path.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Ming reported that with the abort path of the descriptor submission, there
can be a window where a completed descriptor can be missed to be completed
by the irq completion thread:

CPU A				CPU B
Submit (successful)

Submit (fail)
				irq_process_work_list() // empty

llist_abort_desc()
// remove all descs from pending list

				irq_process_pending_llist() // empty
				exit idxd_wq_thread() with no processing

Add opportunistic descriptor completion in the abort path in order to
remove the missed completion.

Fixes: 6b4b87f2 ("dmaengine: idxd: fix submission race window")
Reported-by: NMing Li <ming4.li@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163898288714.443911.16084982766671976640.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit fa51b16d
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit fa51b16d dmaengine: idxd: fix calling wq quiesce inside spinlock.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Dan reports that smatch has found idxd_wq_quiesce() is being called inside
the idxd->dev_lock. idxd_wq_quiesce() calls wait_for_completion() and
therefore it can sleep. Move the call outside of the spinlock as it does
not need device lock.

Fixes: 5b0c68c4 ("dmaengine: idxd: support reporting of halt interrupt")
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163716858508.1721911.15051495873516709923.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit a3e340c1
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit a3e340c1 dmaengine: idxd: fix resource leak on dmaengine driver disable.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The wq resources needs to be released before the kernel type is reset by
__drv_disable_wq(). With dma channels unregistered and wq quiesced, all the
wq resources for dmaengine can be freed. There is no need to wait until wq
is disabled. With the wq->type being reset to "unknown", the driver is
skipping the freeing of the resources.

Fixes: 0cda4f69 ("dmaengine: idxd: create dmaengine driver for wq 'device'")
Reported-by: NJacob Pan <jacob.jun.pan@linux.intel.com>
Tested-by: NJacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163517405099.3484556.12521975053711345244.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit 2efe58cf
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 2efe58cf dmaengine: idxd: cleanup completion record allocation.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

According to core-api/dma-api-howto.rst, the address from
dma_alloc_coherent is gauranteed to align to the smallest PAGE_SIZE order.
That supercedes the 64B/32B alignment requirement of the completion record.
Remove alignment adjustment code.
Tested-by: NJacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163517396063.3484297.7494385225280705372.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit e530a9f3
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit e530a9f3 dmaengine: idxd: reconfig device after device reset command.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Device reset clears the MSIXPERM table and the device registers. Re-program
the MSIXPERM table and re-enable the error interrupts post reset.

Fixes: 745e92a6 ("dmaengine: idxd: idxd: move remove() bits for idxd 'struct device' to device.c")
Reported-by: NSanjay Kumar <sanjay.k.kumar@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163054188513.2853562.12077053294595278181.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit 88d97ea8
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 88d97ea8 dmaengine: idxd: add halt interrupt support.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Add halt interrupt support. Given that the misc interrupt handler already
check halt state, the driver just need to run the halt handling code when
receiving the halt interrupt.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163114224352.846654.14334468363464318828.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit ee5c6f0c
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit ee5c6f0c dmaengine: idxd: Use list_move_tail instead of list_del/list_add_tail.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Using list_move_tail() instead of list_del() + list_add_tail()
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NBixuan Cui <cuibixuan@huawei.com>
Acked-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20210908092826.67765-1-cuibixuan@huawei.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit 98da0106
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 98da0106 dmanegine: idxd: fix resource free ordering on driver removal.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Fault triggers on ioread32() when pci driver unbind is envoked. The
placement of idxd sub-driver removal causes the probing of the device mmio
region after the mmio mapping being torn down. The driver needs the
sub-drivers to be unbound but not release the idxd context until all
shutdown activities has been done. Move the sub-driver unregistering up
before the remove() calls shutdown(). But take a device ref on the
idxd->conf_dev so that the memory does not get freed in ->release(). When
all cleanup activities has been done, release the ref to allow the idxd
memory to be freed.

[57159.542766] RIP: 0010:ioread32+0x27/0x60
[57159.547097] Code: 00 66 90 48 81 ff ff ff 03 00 77 1e 48 81 ff 00 00 01 00 76 05 0f
 b7 d7 ed c3 8b 15 03 50 41 01 b8 ff ff ff ff 85 d2 75 04 c3 <8b> 07 c3 55 83 ea 01 48
 89 fe 48 c7 c7 00 70 5f 82 48 89 e5 48 83
[57159.566647] RSP: 0018:ffffc900011abb60 EFLAGS: 00010292
[57159.572295] RAX: ffffc900011e0000 RBX: ffff888107d39800 RCX: 0000000000000000
[57159.579842] RDX: 0000000000000000 RSI: ffffffff82b1e448 RDI: ffffc900011e0090
[57159.587421] RBP: ffffc900011abb88 R08: 0000000000000000 R09: 0000000000000001
[57159.594972] R10: 0000000000000001 R11: 0000000000000000 R12: ffff8881019840d0
[57159.602533] R13: ffff8881097e9000 R14: ffffffffa08542a0 R15: 00000000000003a8
[57159.610093] FS:  00007f991e0a8740(0000) GS:ffff888459900000(0000) knlGS:00000000000
00000
[57159.618614] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[57159.624814] CR2: ffffc900011e0090 CR3: 000000010862a002 CR4: 00000000003706e0
[57159.632397] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[57159.639973] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[57159.647601] Call Trace:
[57159.650502]  ? idxd_device_disable+0x41/0x110 [idxd]
[57159.655948]  idxd_device_drv_remove+0x2b/0x80 [idxd]
[57159.661374]  idxd_config_bus_remove+0x16/0x20
[57159.666191]  __device_release_driver+0x163/0x240
[57159.671320]  device_release_driver+0x2b/0x40
[57159.676052]  bus_remove_device+0xf5/0x160
[57159.680524]  device_del+0x19c/0x400
[57159.684440]  device_unregister+0x18/0x60
[57159.688792]  idxd_remove+0x140/0x1c0 [idxd]
[57159.693406]  pci_device_remove+0x3e/0xb0
[57159.697758]  __device_release_driver+0x163/0x240
[57159.702788]  device_driver_detach+0x43/0xb0
[57159.707424]  unbind_store+0x11e/0x130
[57159.711537]  drv_attr_store+0x24/0x30
[57159.715646]  sysfs_kf_write+0x4b/0x60
[57159.719710]  kernfs_fop_write_iter+0x153/0x1e0
[57159.724563]  new_sync_write+0x120/0x1b0
[57159.728812]  vfs_write+0x23e/0x350
[57159.732624]  ksys_write+0x70/0xf0
[57159.736335]  __x64_sys_write+0x1a/0x20
[57159.740492]  do_syscall_64+0x3b/0x90
[57159.744465]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[57159.749908] RIP: 0033:0x7f991e19c387
[57159.753898] Code: 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e
 fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51
 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
[57159.773564] RSP: 002b:00007ffc2ce2d6a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[57159.781550] RAX: ffffffffffffffda RBX: 000000000000000c RCX: 00007f991e19c387
[57159.789133] RDX: 000000000000000c RSI: 000055ee2630e140 RDI: 0000000000000001
[57159.796695] RBP: 000055ee2630e140 R08: 0000000000000000 R09: 00007f991e2324e0
[57159.804246] R10: 00007f991e2323e0 R11: 0000000000000246 R12: 000000000000000c
[57159.811800] R13: 00007f991e26f520 R14: 000000000000000c R15: 00007f991e26f700
[57159.819373] Modules linked in: idxd bridge stp llc bnep sunrpc nls_iso8859_1 intel_
rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_code
c_realtek iTCO_wdt 8250_dw snd_hda_codec_generic kvm_intel ledtrig_audio iTCO_vendor_s
upport snd_hda_intel snd_intel_dspcfg ppdev kvm snd_hda_codec intel_wmi_thunderbolt sn
d_hwdep irqbypass iwlwifi btusb snd_hda_core rapl btrtl intel_cstate snd_seq btbcm snd
_seq_device btintel snd_pcm cfg80211 bluetooth pcspkr psmouse input_leds snd_timer int
el_lpss_pci mei_me intel_lpss snd ecdh_generic ecc mei ucsi_acpi i2c_i801 idma64 i2c_s
mbus virt_dma soundcore typec_ucsi typec wmi parport_pc parport video mac_hid acpi_pad
 sch_fq_codel drm ip_tables x_tables crct10dif_pclmul crc32_pclmul ghash_clmulni_intel
 usbkbd hid_generic usbmouse aesni_intel usbhid crypto_simd cryptd e1000e hid serio_ra
w ahci libahci pinctrl_sunrisepoint fuse msr autofs4 [last unloaded: idxd]
[57159.904082] CR2: ffffc900011e0090
[57159.907877] ---[ end trace b4e32f49ce9176a4 ]---

Fixes: 49c4959f ("dmaengine: idxd: fix sequence for pci driver remove() and shutdown()")
Reported-by: NZiye Yang <ziye.yang@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163225535868.4152687.9318737776682088722.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit 15af8408
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 15af8408 dmaengine: idxd: remove kernel wq type set when load configuration.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Remove setting of wq type on guest kernel during configuration load on RO
device config. The user will set the kernel wq type and this setting based
on config is not necessary.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163474724511.2607444.1876715711451990426.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit c5b64b68
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit c5b64b68 dmaengine: idxd: remove gen cap field per spec 1.2 update.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Remove max_descs_per_engine field. The recently released DSA spec 1.2 [1]
has removed this field and made it reserved.

[1]: https://software.intel.com/content/dam/develop/external/us/en/documents-tps/341204-intel-data-streaming-accelerator-spec.pdfSigned-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163406167978.1303649.1798682437841822837.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit 79c4c3db
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 79c4c3db dmaengine: idxd: check GENCAP config support for gencfg register.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

DSA spec 1.2 has moved the GENCFG register under the GENCAP configuration
support with respect to writability. Add check in driver before writing to
GENCFG register.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163406171896.1303830.11217958011385656998.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.16
commit 85f604af
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 85f604af dmaengine: idxd: move out percpu_ref_exit() to ensure it's outside submission.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

percpu_ref_tryget_live() is safe to call as long as ref is between init and
exit according to the function comment. Move percpu_ref_exit() so it is
called after the dma channel is no longer valid to ensure this holds true.

Fixes: 93a40a6d ("dmaengine: idxd: add percpu_ref to descriptor submission path")
Suggested-by: NKevin Tian <kevin.tian@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163294293832.914350.10326422026738506152.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit cf84a4b9
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit cf84a4b9 dmaengine: idxd: remove interrupt disable for dev_lock.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The spinlock is not being used in hard interrupt context. There is no need
to disable irq when acquiring the lock. The interrupt thread handler also
is not in bottom half context, therefore we can also remove disabling of
the bh. Convert all dev_lock acquisition to plain spin_lock() calls.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162984026772.1939166.11504067782824765879.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit f9f4082d
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit f9f4082d dmaengine: idxd: remove interrupt disable for cmd_lock.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The cmd_lock spinlock is not being used in hard interrupt context. There is
no need to disable irq when acquiring the lock. Convert all cmd_lock
acquisition to plain spin_lock() calls.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162984027930.1939209.15758413737332339204.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit d8071323
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit d8071323 dmaengine: idxd: fix setting up priv mode for dwq.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

DSA spec says WQ priv bit is 0 if the Privileged Mode Enable field of the
PCI Express PASID capability is 0 and pasid is enabled. Make sure that the
WQCFG priv field is set correctly according to usage type. Reject config if
setting up kernel WQ type and no support. Also add the correct priv setup
for a descriptor.

Fixes: 484f910e ("dmaengine: idxd: fix wq config registers offset programming")
Cc: Ramesh Thomas <ramesh.thomas@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162939084657.903168.14160019185148244596.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 9806eb5c
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 9806eb5c dmaengine: idxd: set descriptor allocation size to threshold for swq.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Since submission is sent to limited portal, the actual wq size for shared
wq is set by the threshold rather than the wq size. When the wq type is
shared, set the allocated descriptors to the threshold.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162827151733.3459223.3829837172226042408.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 0b030f54
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 0b030f54 dmaengine: idxd: make submit failure path consistent on desc freeing.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The submission path for dmaengine API does not do descriptor freeing on
failure. Also, with the abort mechanism, the freeing of descriptor happens
when the abort callback is completed. Therefore free descriptor on all
error paths for submission call to make things consistent. Also remove the
double free that would happen on abort in idxd_dma_tx_submit() call.

Fixes: 6b4b87f2 ("dmaengine: idxd: fix submission race window")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162827146072.3459011.10255348500504659810.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 9fce3b3a
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 9fce3b3a dmaengine: idxd: remove interrupt flag for completion list spinlock.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The list lock is never acquired in interrupt context. Therefore there is no
need to disable interrupts. Remove interrupt flags for lock operations.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162826417450.3454650.3733188117742416238.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit d803c8b9
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit d803c8b9 dmaengine: idxd: make I/O interrupt handler one shot.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The interrupt thread handler currently loops forever to process outstanding
completions. This causes either an "irq X: nobody cared" kernel splat or
the NMI watchdog kicks in due to running too long in the function. The irq
thread handler is expected to run again after exiting if there are
interrupts fired while the thread handler is running. So the handler code
can process all the completed I/O in a single pass and exit without losing
the follow on completed I/O.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162802977005.3084234.11836261157026497585.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit bd2f4ae5
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit bd2f4ae5 dmaengine: idxd: clear block on fault flag when clear wq.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The block on fault flag is not cleared when we disable or reset wq. This
causes it to remain set if the user does not clear it on the next
configuration load. Add clear of flag in dxd_wq_disable_cleanup()
routine.

Fixes: da32b28c ("dmaengine: idxd: cleanup workqueue config after disabling")
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162803023553.3086015.8158952172068868803.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 81c2f79c
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 81c2f79c dmaengine: idxd: add capability check for 'block on fault' attribute.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The device general capability has a bit that indicate whether 'block on
fault' is supported. Add check to wq sysfs knob to check if cap exists
before allowing user to toggle.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162802992615.3084999.12539468940404102898.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 53cbf462
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 53cbf462 dmaengine: idxd: Remove unused status variable in irq_process_work_list().
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

status is no longer used within this block:

drivers/dma/idxd/irq.c:255:6: warning: unused variable 'status'
[-Wunused-variable]
                u8 status = desc->completion->status & DSA_COMP_STATUS_MASK;
                   ^
1 warning generated.

Fixes: b60bb6e2 ("dmaengine: idxd: fix abort status check")
Signed-off-by: NNathan Chancellor <nathan@kernel.org>
Acked-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/20210802175820.3153920-1-nathan@kernel.orgSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit e9c5b0b5
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit e9c5b0b5 dmaengine: idxd: Fix a possible NULL pointer dereference.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

'device_driver_attach()' dereferences its first argument (i.e. 'alt_drv')
so it must not be NULL.
Simplify the error handling logic about NULL 'alt_drv' in order to be
more robust and future-proof.

Fixes: 568b2126 ("dmaengine: idxd: fix uninit var for alt_drv")
Fixes: 6e7f3ee9 ("dmaengine: idxd: move dsa_drv support to compatible mode")
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Acked-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/77f0dc4f3966591d1f0cffb614a94085f8895a85.1627560174.git.christophe.jaillet@wanadoo.frSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit b60bb6e2
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit b60bb6e2 dmaengine: idxd: fix abort status check.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Coverity static analysis of linux-next found issue.

The check (status == IDXD_COMP_DESC_ABORT) is always false since status
was previously masked with 0x7f and IDXD_COMP_DESC_ABORT is 0xff.

Fixes: 6b4b87f2 ("dmaengine: idxd: fix submission race window")
Reported-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162698465160.3560828.18173186265683415384.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 125d1037
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 125d1037 dmanegine: idxd: add software command status.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Enabling device and wq returns standard errno and that does not provide
enough details to indicate what exactly failed. The hardware command status
is only 8bits. Expand the command status to 32bits and use the upper 16
bits to define software errors to provide more details on the exact
failure. Bit 31 will be used to indicate the error is software set as the
driver is using some of the spec defined hardware error as well.

Cc: Ramesh Thomas <ramesh.thomas@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162681373579.1968485.5891788397526827892.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit a9c17152
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit a9c17152 dmaengine: idxd: rotate portal address for better performance.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The device submission portal is on a 4k page and any of those 64bit aligned
address on the page can be used for descriptor submission. By rotating the
offset through the 4k range and prevent successive writes to the same MMIO
address, performance improvement is observed through testing.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162681372446.1968485.10634280461681015569.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 568b2126
category: bugfix
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 568b2126 dmaengine: idxd: fix uninit var for alt_drv.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

0-day detected uninitialized alt_drv variable in the bind_store() function.
The branch can be taken when device is not idxd device or wq 'struct
device'. Init alt_drv to NULL.

Fixes: 6e7f3ee9 ("dmaengine: idxd: move dsa_drv support to compatible mode")
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162689250332.2114335.636367120454420852.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit ade8a86b
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit ade8a86b dmaengine: idxd: Set defaults for GRPCFG traffic class.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Set GRPCFG traffic class to value of 1 for best performance on current
generation of accelerators. Also add override option to allow experimentation.
Sysfs knobs are disabled for DSA/IAX gen1 devices.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162681373005.1968485.3761065664382799202.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 0e96454c
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 0e96454c dmaengine: idxd: remove fault processing code.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Kernel memory are pinned and will not cause faults. Since the driver
does not support interrupts for user descriptors, no fault errors are
expected to come through the misc interrupt. Remove dead code.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162630502789.631986.10591230961790023856.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 6e7f3ee9
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/i596wo
cve: na

Intel-SIG: commit 6e7f3ee9 dmaengine: idxd: move dsa_drv support to compatible mode.
incremental backporting patches for dsa/iaa on intel xeon platform.

--------------------------------

The original architecture of /sys/bus/dsa invented a scheme whereby
a single entry in the list of bus drivers, /sys/bus/drivers/dsa,
handled all device types and internally routed them to different
different drivers. Those internal drivers were invisible to
userspace.

With the idxd driver transitioned to a proper bus device-driver model,
the legacy behavior needs to be preserved due to it being exposed to
user space via sysfs. Create a compat driver to provide the legacy
behavior for /sys/bus/dsa/drivers/dsa. This should satisfy user
tool accel-config v3.2 or ealier where this behavior is expected.
If the distro has a newer accel-config then the legacy mode does
not need to be enabled.

When the compat driver binds the device (i.e. dsa0) to the dsa driver,
it will be bound to the new idxd_drv. The wq device (i.e. wq0.0) will
be bound to either the dmaengine_drv or the user_drv. The dsa_drv
becomes a routing mechansim for the new drivers. It will not support
additional external drivers that are implemented later.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637468705.744545.4399080971745974435.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit d9e5481f
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit d9e5481f dmaengine: dsa: move dsa_bus_type out of idxd driver to standalone.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

In preparation for dsa_drv compat support to be built-in, move the bus
code to its own compilation unit. A follow-on patch adds the compat
implementation. Recall that the compat implementation allows for the
deprecated / omnibus dsa_drv binding scheme rather than the idiomatic
organization of a full fledged bus driver per driver type.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637468142.744545.2811632736881720857.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 448c3de8
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 448c3de8 dmaengine: idxd: create user driver for wq 'device'.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The original architecture of /sys/bus/dsa invented a scheme whereby a
single entry in the list of bus drivers, /sys/bus/drivers/dsa, handled
all device types and internally routed them to different drivers.
Those internal drivers were invisible to userspace. Now, as
/sys/bus/dsa wants to grow support for alternate drivers for a given
device, for example vfio-mdev instead of kernel-internal-dmaengine, a
proper bus device-driver model is needed. The first step in that process
is separating the existing omnibus/implicit "dsa" driver into proper
individual drivers registered on /sys/bus/dsa. Establish the
idxd_user_drv driver that controls the enabling and disabling of the
wq and also register and unregister a char device to allow user space
to mmap the descriptor submission portal.

The cdev related bits are moved to the cdev driver probe/remove and out of
the drv_enabe/disable_wq() calls. These bits are exclusive to the cdev
operation and not part of the generic enable/disable of the wq device.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637467578.744545.10203997610072341376.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 0cda4f69
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 0cda4f69 dmaengine: idxd: create dmaengine driver for wq 'device'.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The original architecture of /sys/bus/dsa invented a scheme whereby a
single entry in the list of bus drivers, /sys/bus/drivers/dsa, handled
all device types and internally routed them to different drivers.
Those internal drivers were invisible to userspace. Now, as
/sys/bus/dsa wants to grow support for alternate drivers for a given
device, for example vfio-mdev instead of kernel-internal-dmaengine, a
proper bus device-driver model is needed. The first step in that process
is separating the existing omnibus/implicit "dsa" driver into proper
individual drivers registered on /sys/bus/dsa. Establish the
idxd_dmaengine_drv driver that controls the enabling and disabling of the
wq and also register and unregister the dma channel.

idxd_wq_alloc_resources() and idxd_wq_free_resources() also get moved to
the dmaengine driver. The resources (dma descriptors allocation and setup)
are only used by the dmaengine driver and should only happen when it loads.

The char dev driver (cdev) related bits are left in the __drv_enable_wq()
and __drv_disable_wq() calls to be moved when we split out the char dev
driver just like how the dmaengine driver is split out.

WQ autoload support is not expected currently. With the amount of
configuration needed for the device, the wq is always expected to
be enabled by a tool (or via sysfs) rather than auto enabled at driver
load.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637467033.744545.12330636655625405394.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 034b3290
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 034b3290 dmaengine: idxd: create idxd_device sub-driver.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

The original architecture of /sys/bus/dsa invented a scheme whereby a
single entry in the list of bus drivers, /sys/bus/drivers/dsa, handled
all device types and internally routed them to different drivers.
Those internal drivers were invisible to userspace. Now, as
/sys/bus/dsa wants to grow support for alternate drivers for a given
device, for example vfio-mdev instead of kernel-internal-dmaengine, a
proper bus device-driver model is needed. The first step in that process
is separating the existing omnibus/implicit "dsa" driver into proper
individual drivers registered on /sys/bus/dsa. Establish the idxd_drv
driver that control the enabling and disabling of the accelerator device.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637466439.744545.15210886092627144577.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 5fee6567
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 5fee6567 dmaengine: idxd: add type to driver in order to allow device matching.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Add an array of support device types to the idxd_device_driver
definition in order to enable simple matching of device type to a
given driver. The deprecated / omnibus dsa_drv driver specifies
IDXD_DEV_NONE as its only role is to service legacy userspace (old
accel-config) directed bind requests and route them to them the proper
driver. It need not attach to a device when the bus is autoprobed. The
accel-config tooling is being updated to drop its dependency on this
deprecated bind scheme.
Reviewed-by: NDan Willliams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637465882.744545.17456174666211577867.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit c05257b5
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit c05257b5 dmanegine: idxd: open code the dsa_drv registration.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Don't need a wrapper to register the driver. Just do it directly.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637465319.744545.16325178432532362906.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit 745e92a6
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit 745e92a6 dmaengine: idxd: idxd: move remove() bits for idxd 'struct device' to device.c.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Move the code related to a ->remove() function for the idxd
'struct device' to device.c to prep for the idxd device
sub-driver in device.c.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637464768.744545.15797285510999151668.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit bd42805b
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit bd42805b dmaengine: idxd: move probe() bits for idxd 'struct device' to device.c.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Move the code related to a ->probe() function for the idxd
'struct device' to device.c to prep for the idxd device
sub-driver in device.c.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637464189.744545.17423830646786162194.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>

mainline inclusion
from mainline-v5.15
commit fcc2281b
category: feature
bugzilla: https://gitee.com/openeuler/intel-kernel/issues/I596WO
CVE: NA

Intel-SIG: commit fcc2281b dmaengine: idxd: fix bus_probe() and bus_remove() for dsa_bus.
Incremental backporting patches for DSA/IAA on Intel Xeon platform.

--------------------------------

Current implementation have put all the code that should be in a driver
probe/remove in the bus probe/remove function. Add ->probe() and ->remove()
support for the dsa_drv and move all those code out of bus probe/remove.
The change does not split out the distinction between device sub-driver and
wq sub-driver. It only cleans up the bus calls. The split out will be
addressed in follow on patches.
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/162637463586.744545.5806250155539938643.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NVinod Koul <vkoul@kernel.org>
Signed-off-by: NXiaochen Shen <xiaochen.shen@intel.com>