Just init curr_sgl_dma = 0 to avoid compile warning.

Fixes: dfed0098 ("crypto: hisilicon - add hardware SGL support")
Reported-by: Nkbuild test robot <lkp@intel.com>
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add ARM64/PCI/PCI_MSI dependency for CRYPTO_DEV_HISI_ZIP.

Fixes: 62c455ca ("crypto: hisilicon - add HiSilicon ZIP accelerator support")
Reported-by: Nkbuild test robot <lkp@intel.com>
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix to use proper type of argument for dma_addr_t and size_t.

Fixes: 263c9959 ("crypto: hisilicon - add queue management driver for HiSilicon QM module")
Reported-by: Nkbuild test robot <lkp@intel.com>
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Exercising CPU hotplug on a 5.2 kernel with recent padata fixes from
cryptodev-2.6.git in an 8-CPU kvm guest...

    # modprobe tcrypt alg="pcrypt(rfc4106(gcm(aes)))" type=3
    # echo 0 > /sys/devices/system/cpu/cpu1/online
    # echo c > /sys/kernel/pcrypt/pencrypt/parallel_cpumask
    # modprobe tcrypt mode=215

...caused the following crash:

    BUG: kernel NULL pointer dereference, address: 0000000000000000
    #PF: supervisor read access in kernel mode
    #PF: error_code(0x0000) - not-present page
    PGD 0 P4D 0
    Oops: 0000 [#1] SMP PTI
    CPU: 2 PID: 134 Comm: kworker/2:2 Not tainted 5.2.0-padata-base+ #7
    Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-<snip>
    Workqueue: pencrypt padata_parallel_worker
    RIP: 0010:padata_reorder+0xcb/0x180
    ...
    Call Trace:
     padata_do_serial+0x57/0x60
     pcrypt_aead_enc+0x3a/0x50 [pcrypt]
     padata_parallel_worker+0x9b/0xe0
     process_one_work+0x1b5/0x3f0
     worker_thread+0x4a/0x3c0
     ...

In padata_alloc_pd, pd->cpu is set using the user-supplied cpumask
instead of the effective cpumask, and in this case cpumask_first picked
an offline CPU.

The offline CPU's reorder->list.next is NULL in padata_reorder because
the list wasn't initialized in padata_init_pqueues, which only operates
on CPUs in the effective mask.

Fix by using the effective mask in padata_alloc_pd.

Fixes: 6fc4dbcf ("padata: Replace delayed timer with immediate workqueue in padata_reorder")
Signed-off-by: NDaniel Jordan <daniel.m.jordan@oracle.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Steffen Klassert <steffen.klassert@secunet.com>
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Remove unused includes of linux/pci.h.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NGary R Hook <gary.hook@amd.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

ccp-dev.h uses dma_direction, which is defined in linux/dma-direction.h.
Include that explicitly instead of relying on it being included via
linux/pci.h, since ccp-dev.h requires nothing else from linux/pci.h.

Similarly, ccp-dmaengine.c uses dma_get_mask(), which is defined in
linux/dma-mapping.h, so include that explicitly since it requires nothing
else from linux/pci.h.

A future patch will remove the includes of linux/pci.h where it is not
needed.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NGary R Hook <gary.hook@amd.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Reviewed-by: NHeiko Stuebner <heiko@sntech.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Acked-by: NJamie Iles <jamie@jamieiles.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Acked-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Reviewed-by: NKrzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use devm_platform_ioremap_resource() to simplify the code a bit.
This is detected by coccinelle.
Reported-by: NHulk Robot <hulkci@huawei.com>
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Based on seqiv, IPsec ESP and rfc4543/rfc4106 the assoclen can be 16 or
20 bytes.

From esp4/esp6, assoclen is sizeof IP Header. This includes spi, seq_no
and extended seq_no, that is 8 or 12 bytes.
In seqiv, to asscolen is added the IV size (8 bytes).
Therefore, the assoclen, for rfc4543, should be restricted to 16 or 20
bytes, as for rfc4106.
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geanta <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add Zhou Wang as a maintainer for HiSilicon QM and ZIP controller driver.
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

HiSilicon ZIP engine driver uses debugfs to provide debug information,
the usage can be found in /Documentation/ABI/testing/debugfs-hisi-zip.
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add debugfs descriptions for HiSilicon ZIP and QM driver.
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Reviewed-by: NJonathan Cameron <Jonathan.Cameron@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

HiSilicon ZIP engine supports PCI SRIOV. This patch enable this feature.
User can enable VFs and pass through them to VM, same ZIP driver can work
in VM to provide ZLIB and GZIP algorithm by crypto acomp interface.
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The HiSilicon ZIP accelerator implements the zlib and gzip algorithm. It
uses Hisilicon QM as the interface to the CPU.

This patch provides PCIe driver to the accelerator and registers it to
crypto acomp interface. It also uses sgl as data input/output interface.
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NShiju Jose <shiju.jose@huawei.com>
Signed-off-by: NKenneth Lee <liguozhu@hisilicon.com>
Signed-off-by: NHao Fang <fanghao11@huawei.com>
Reviewed-by: NJonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

HiSilicon accelerators in Hip08 use same hardware scatterlist for data format.
We support it in this module.

Specific accelerator drivers can use hisi_acc_create_sgl_pool to allocate
hardware SGLs ahead. Then use hisi_acc_sg_buf_map_to_hw_sgl to get one
hardware SGL and pass related information to hardware SGL.

The DMA address of mapped hardware SGL can be passed to SGL src/dst field
in QM SQE.
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

QM is a general IP used by HiSilicon accelerators. It provides a general
PCIe interface for the CPU and the accelerator to share a group of queues.

A QM integrated in an accelerator provides queue management service.
Queues can be assigned to PF and VFs, and queues can be controlled by
unified mailboxes and doorbells. Specific task request are descripted by
specific description buffer, which will be controlled and pass to related
accelerator IP by QM.

This patch adds a QM driver used by the accelerator driver to access
the QM hardware.
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NKenneth Lee <liguozhu@hisilicon.com>
Signed-off-by: NShiju Jose <shiju.jose@huawei.com>
Signed-off-by: NHao Fang <fanghao11@huawei.com>
Reviewed-by: NJonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The crypto engine initializes its kworker thread to FIFO-99 (when
requesting RT priority), reduce this to FIFO-50.

FIFO-99 is the very highest priority available to SCHED_FIFO and
it not a suitable default; it would indicate the crypto work is the
most important work on the machine.

Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: NPeter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

To be consistent with other CAAM modules, caamhash should return 0
instead of -ENODEV in case CAAM has no MDHA.

Based on commit 1b46c90c ("crypto: caam - convert top level drivers to libraries")
the value returned by entry point is never checked and
the exit point is always executed.
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geanta <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

To know if a registration succeeded added a new struct,
caam_akcipher_alg, that keeps, also, the registration status.
This status is updated in caam_pkc_init and verified in
caam_pkc_exit to unregister an algorithm.

Fixes: 1b46c90c ("crypto: caam - convert top level drivers to libraries")
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geanta <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit 1b46c90c ("crypto: caam - convert top level drivers to libraries")
changed entry and exit points behavior for caamalg,
caamalg_qi, caamalg_qi2, caamhash, caampkc, caamrng.

For example, previously caam_pkc_init() and caam_pkc_exit() were
module entry/exit points. This means that if an error would happen
in caam_pkc_init(), then caam_pkc_exit() wouldn't have been called.
After the mentioned commit, caam_pkc_init() and caam_pkc_exit()
are manually called - from jr.c. caam_pkc_exit() is called
unconditionally, even if caam_pkc_init() failed.

Added a global variable to keep the status of the algorithm
registration and free of resources.
The exit point of caampkc/caamrng module is executed only if the
registration was successful. Therefore we avoid double free of
resources in case the algorithm registration failed.

Fixes: 1b46c90c ("crypto: caam - convert top level drivers to libraries")
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geanta <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Check the return value of the hardware registration for caam_rng and free
resources in case of failure.

Fixes: e24f7c9e ("crypto: caam - hwrng support")
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geanta <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fuzz testing uncovered an issue when |user key| > |derived key|.
Derived key generation has to be fixed in two cases:

1. Era >= 6 (DKP is available)
DKP cannot be used with immediate input key if |user key| > |derived key|,
since the resulting descriptor (after DKP execution) would be invalid -
having a few bytes from user key left in descriptor buffer
as incorrect opcodes.

Fix DKP usage both in standalone hmac and in authenc algorithms.
For authenc the logic is simplified, by always storing both virtual
and dma key addresses.

2. Era < 6
The same case (|user key| > |derived key|) fails when DKP
is not available.
Make sure gen_split_key() dma maps max(|user key|, |derived key|),
since this is an in-place (bidirectional) operation.
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Reviewed-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Update alginfo struct to keep both virtual and dma key addresses,
so that descriptors have them at hand.
One example where this is needed is in the xcbc(aes) shared descriptors,
which are updated in current patch.
Another example is the upcoming fix for DKP.
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Reviewed-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Update share descriptor for rfc4106 to skip instructions in case
cryptlen is zero. If no instructions are jumped the DECO hangs and a
timeout error is thrown.
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geanta <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Check zero-length input, for skcipher algorithm, to solve the extra
tests. This is a valid operation, therefore the API will return no error.
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geanta <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Check assoclen to solve the extra tests that expect -EINVAL to be
returned when the associated data size is not valid.

Validated assoclen for RFC4106 and RFC4543 which expects an assoclen
of 16 or 20.
Based on seqiv, IPsec ESP and RFC4543/RFC4106 the assoclen is sizeof IP
Header (spi, seq_no, extended seq_no) and IV len. This can be 16 or 20
bytes.
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geanta <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Check authsize to solve the extra tests that expect -EINVAL to be
returned when the authentication tag size is not valid.

Validated authsize for GCM, RFC4106 and RFC4543.
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geanta <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Check key length to solve the extra tests that expect -EINVAL to be
returned when the key size is not valid.

Validated AES keylen for skcipher, ahash and aead.
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geanta <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

skcipher encryption might fail and in some cases, like (invalid) input
length smaller then block size, updating the IV would lead to a useless
IV copy in case hardware issued an error.
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Modify drive to provide a valid errno (and not the HW error ID)
to the user, via completion callbacks.

A "valid errno" is currently not explicitly mentioned in the docs,
however the error code is expected to match the one returned by the
generic SW implementation.

Note: in most error cases caam/qi and caam/qi2 returned -EIO; align all
caam drivers to return -EINVAL.

While here, ratelimit prints triggered by fuzz testing, such that
console is not flooded.
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>