Since we are going to introduce VFIO PCI HiSilicon ACC driver for live
migration in subsequent patches, move the ACC QM header file to a
common include dir.
Acked-by: NZhou Wang <wangzhou1@hisilicon.com>
Acked-by: NLongfang Liu <liulongfang@huawei.com>
Acked-by: NKai Ye <yekai13@huawei.com>
Signed-off-by: NShameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Link: https://lore.kernel.org/r/20220308184902.2242-2-shameerali.kolothum.thodi@huawei.comSigned-off-by: NAlex Williamson <alex.williamson@redhat.com>

Add runtime PM support for Kunpeng930 accelerator device.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Accelerator devices support runtime PM to reduce power consumption.
This patch adds the runtime PM suspend/resume callbacks to the
accelerator devices.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The accelerator devices support runtime PM, when device is in suspended, an
exception will occur if reading registers. Therefore, this patch uses
'debugfs_create_file' instead of 'debugfs_create_regset32' to create
debugfs file, and then the driver can get the device status before
reading the register.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Based on the Token bucket algorithm. The HAC driver supports to configure
each function's QoS in the host. The driver supports writing QoS by the
debugfs node that named "alg_qos". The qos value is 1~1000.
Signed-off-by: NKai Ye <yekai13@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Kunpeng930 hardware supports the communication between PF and VFs.

This patch enables communication between PF and VFs by writing hardware
registers, and requests an irq for communication.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Kunpeng930 hardware supports address prefetching to improve performance
before doing tasks in SVA scenario.

This patch enables this function in device initialization by writing
hardware registers. In the process of reset, address prefetching is
disabled to avoid the failure of interaction between accelerator device
and SMMU.
Signed-off-by: NLongfang Liu <liulongfang@huawei.com>
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

QM adds 'qm_mailbox_timeout' and 'qm_flr_timeout' hardware error types on
Kunpeng930. This patch enables the new error types and configures the error
types as NFE.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

'err_info' does not support dynamic configuration since it is const type.
Therefore, in order to support new error type later, 'err_info' is changed
to dynamic configuration.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Kunpeng930 supports doorbell isolation to ensure that each queue
has an independent doorbell address space.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Kunpeng930 supports queue doorbell isolation.
When doorbell isolation is enabled, it supports to obtain the
maximum number of queues of one function from hardware register.
Otherwise, the 'max_qp_num' is the total number of queues.

When assigning queues to VF, it is necessary to ensure that the number
of VF queues does not exceed 'max_qp_num'.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Since the code related to 'CURRENT_QM' debugfs is exactly same in
sec/hpre/zip driver, move 'CURRENT_QM' to qm.c to reduce duplicate code.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

A new generation of accelerator Kunpeng930 has appeared, and the
corresponding driver needs to be updated to support some new
algorithms of Kunpeng930. To be compatible with Kunpeng920, we
add parameter 'struct hisi_qm *qm' to sec_algs_(un)register to
identify the chip's version.
Signed-off-by: NMeng Yu <yumeng18@huawei.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Reviewed-by: NLongfang Liu <liulongfang@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes inconsistent of printing format with argument type.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

There is no need to reset hardware when Corrected Error(CE) happens.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add waiting logic for resetting as removing driver,
otherwise call trace will occur due to releasing resource.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Enable PASID by setting 'sqc' and 'cqc' pasid bits
per queue in Kunpeng 930.

For Kunpeng 920, PASID is effective for all queues once set
in SVA scenarios.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NHui Tang <tanghui20@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add 'uacce_mode' parameter for ZIP, which can be set as 0(default) or 1.
'0' means ZIP is only registered to kernel crypto, and '1' means it's
registered to both kernel crypto and UACCE.
Signed-off-by: NKai Ye <yekai13@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Since 'qm_create_debugfs_file' always returns 0, change it as 'void'.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

At present, as HPRE/SEC/ZIP modules' parameter 'pf_q_num' is 1,
kernel CRYPTO test will fail on the algorithms from the modules,
since 'QP' hardware resources are not enough for CRYPTO TFM.
To fix this, the minimum value of 'pf_q_num' should be 2.
Signed-off-by: NLongfang Liu <liulongfang@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When the devices are removed or not existing, the corresponding algorithms
which are registered by 'hisi-zip' driver can't be used.

Move 'hisi_zip_register_to_crypto' from 'hisi_zip_init' to
'hisi_zip_probe'. The algorithms will be registered to crypto only when
there is device bind on the driver. And when the devices are removed,
the algorithms will be unregistered.

In the previous process, the function 'xxx_register_to_crypto' need a lock
and a static variable to judge if the registration is the first time.
Move this action into the function 'hisi_qm_alg_register'. Each device
will call 'hisi_qm_alg_register' to add itself to qm list in probe process
and registering algs when the qm list is empty.
Signed-off-by: NYang Shen <shenyang39@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Call trace will appear in the Hisilicon crypto driver unbinding or
disabling SRIOV during task running with TFMs on the corresponding
function.
The log looks like this:
	[  293.908078] Call trace:
	[  293.908080]  __queue_work+0x494/0x548
	[  293.908081]  queue_work_on+0x84/0xd8
	[  293.908092]  qm_irq+0x4c/0xd0 [hisi_qm]
	[  293.908096]  __handle_irq_event_percpu+0x74/0x2a0
	[  293.908098]  handle_irq_event_percpu+0x40/0x98
	[  293.908099]  handle_irq_event+0x4c/0x80
	[  293.908101]  handle_fasteoi_irq+0xb0/0x170
	[  293.908102]  generic_handle_irq+0x3c/0x58
	[  293.908103]  __handle_domain_irq+0x68/0xc0
	[  293.908104]  gic_handle_irq+0xb4/0x298
	[  293.908105]  el1_irq+0xcc/0x180
	[  293.908107]  arch_cpu_idle+0x38/0x228
	[  293.908110]  default_idle_call+0x20/0x40
	[  293.908113]  do_idle+0x1cc/0x2b8
	[  293.908114]  cpu_startup_entry+0x2c/0x30
	[  293.908115]  rest_init+0xdc/0xe8
	[  293.908117]  arch_call_rest_init+0x14/0x1c
	[  293.908117]  start_kernel+0x490/0x4c4

This patch adds a waiting logic as user doing the above two operations
to avoid panic. The two operations will hold on in the driver
remove function until the tasks release all their relative TFMs.
Signed-off-by: NHui Tang <tanghui20@huawei.com>
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NYang Shen <shenyang39@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Since the drivers such as HPRE/SEC/ZIP do not implement
'pci_driver.shutdow', a RAS will be triggered at OS rebooting or shutting
down as the hardware device is processing request.
The log looks like this:
	NOTICE:  [NimbusSecNodeType1]:[2372L]This is sec, Base = 0x141800000
	NOTICE:  [NimbusSecHandle]:[2319L] SecIntSt = 0x3
	NOTICE:  [NimbusSecHandle]:[2320L] SecQmIntStatus = 0x2
	NOTICE:  [PrintSecurityType]:[344L] SecurityType is RECOVERABLE!

This patch offers a new API in qm to shutdown devices, and add shutdown
callbacks in ACC driver based on this new API.

So the running devices will be stopped when the OS reboot or shutdown.
Signed-off-by: NYang Shen <shenyang39@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Now, there are three reasons of stopping: 'NORMAL', 'SOFT_RESET' and 'FLR'.
In order to keep this, explicitly pass the stop reason as an input
parameter of 'hisi_qm_stop' function.

Fixes: b67202e8("crypto: hisilicon/qm - add state machine for QM")
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Signed-off-by: NYang Shen <shenyang39@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Updates the initialization and reset of SEC driver's
register operation.
Signed-off-by: NLongfang Liu <liulongfang@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In order to be compatible with devices of different versions, V1 in the
accelerator driver is now isolated, and other versions are the previous
V2 processing flow.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add dump information of SQC/CQC/EQC/AEQC/SQE/CQE/EQE/AEQE.
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add DebugFS method to get the information of IRQ/Requests/QP .etc of QM
for HPRE/ZIP/SEC drivers.
Signed-off-by: NLongfang Liu <liulongfang@huawei.com>
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

By configuring the device error in firmware to report through abnormal
interruption, process all NFE errors in irq handler.
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The hardware device can be configured to report directly through MSI, but
this method will not go through RAS, configure all hardware errors that
should be processed by driver to NFE.
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Put all the code for the memory allocation into the QM initialization
process. Before, The qp memory was allocated when the qp was created,
and released when the qp was released, It is now changed to allocate
all the qp memory once.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The codes related use_dma_api is useless which should be removed.
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add callback reset_prepare and reset_done in QM, The callback
reset_prepare will uninit device error configuration and stop
the QM, the callback reset_done will init the device error
configuration and restart the QM.

Uninit the error configuration will disable device block master OOO
when Multi-bit ECC error occurs to avoid the request of FLR will not
return.
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add specific states for qm and qp, every state change under critical region
to prevent from race condition. Meanwhile, qp state change will also depend
on qm state.

Due to the introduction of these states, it is necessary to pay attention
to the calls of public logic, such as concurrent scenarios resetting and
releasing queue will call hisi_qm_stop, which needs to add additional
status to distinguish and process.
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

put q_num_set similar code into qm to reduce the redundancy.
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add the main implementation of the controller reset interface, which is
roughly divided into three parts, stop, reset, and reinitialization.
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The vfs_num module parameter has been used in zip driver, this patch adds
this for HPRE and SEC driver.
Signed-off-by: NHao Fang <fanghao11@huawei.com>
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Clean the duplicate SR-IOV related codes, put all into qm.c.
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

We plan to move vfs_num related code into qm.c, put the param
vfs_num into struct hisi_qm first.
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use struct hisi_qm to maintain device list. Meanwhile, add two external
interface into qm, merge find proper device and create qp into QP alloc
logic, merge release qps into QP free logic.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>