'qm_eq_ctx_cfg' initializes configuration of EQ and AEQ,
split it into two pieces to improve code readability.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

'qm_qp_ctx_cfg' initializes configuration of SQ and CQ,
split it into two pieces to improve code readability.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Replace 'sprintf' with 'scnprintf' to avoid overrun.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Since 'qm_set_sqctype' always returns 0, change it as 'void'.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Since 'qm_create_debugfs_file' always returns 0, change it as 'void'.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The returns of 'qm_get_hw_error_status' and 'qm_get_dev_err_status'
are values from the hardware registers, which should not be defined
as 'int', so update as 'u32'.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Some numbers are replaced by macros to avoid incomprehension.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Clean up the check for irq. dev_err() is superfluous as
platform_get_irq() already prints an error. Check for zero
would indicate a bug. Remove curly braces to conform to
styling requirements.
Signed-off-by: NNigel Christian <nigel.l.christian@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Loading the module deadlocks since:
-local cbc(aes) implementation needs a fallback and
-crypto API tries to find one but the request_module() resolves back to
the same module

Fix this by changing the module alias for cbc(aes) and
using the NEED_FALLBACK flag when requesting for a fallback algorithm.

Fixes: 00b99ad2 ("crypto: arm/aes-neonbs - Use generic cbc encryption path")
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

A semicolon is not needed after a switch statement.
Signed-off-by: NTom Rix <trix@redhat.com>
Signed-off-by: NTom Rix <trix@redhat.com>
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

A semicolon is not needed after a switch statement.
Signed-off-by: NTom Rix <trix@redhat.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

PAC pointer authentication signs the return address against the value
of the stack pointer, to prevent stack overrun exploits from corrupting
the control flow. However, this requires that the AUTIASP is issued with
SP holding the same value as it held when the PAC value was generated.
The Poly1305 NEON code got this wrong, resulting in crashes on PAC
capable hardware.

Fixes: f569ca16 ("crypto: arm64/poly1305 - incorporate OpenSSL/CRYPTOGAMS ...")
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit 3f69cc60 ("crypto: af_alg - Allow arbitrarily long algorithm
names") made the kernel start accepting arbitrarily long algorithm names
in sockaddr_alg.  However, the actual length of the salg_name field
stayed at the original 64 bytes.

This is broken because the kernel can access indices >= 64 in salg_name,
which is undefined behavior -- even though the memory that is accessed
is still located within the sockaddr structure.  It would only be
defined behavior if the array were properly marked as arbitrary-length
(either by making it a flexible array, which is the recommended way
these days, or by making it an array of length 0 or 1).

We can't simply change salg_name into a flexible array, since that would
break source compatibility with userspace programs that embed
sockaddr_alg into another struct, or (more commonly) declare a
sockaddr_alg like 'struct sockaddr_alg sa = { .salg_name = "foo" };'.

One solution would be to change salg_name into a flexible array only
when '#ifdef __KERNEL__'.  However, that would keep userspace without an
easy way to actually use the longer algorithm names.

Instead, add a new structure 'sockaddr_alg_new' that has the flexible
array field, and expose it to both userspace and the kernel.
Make the kernel use it correctly in alg_bind().

This addresses the syzbot report
"UBSAN: array-index-out-of-bounds in alg_bind"
(https://syzkaller.appspot.com/bug?extid=92ead4eb8e26a26d465e).

Reported-by: syzbot+92ead4eb8e26a26d465e@syzkaller.appspotmail.com
Fixes: 3f69cc60 ("crypto: af_alg - Allow arbitrarily long algorithm names")
Cc: <stable@vger.kernel.org> # v4.12+
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use the new crypto_engine_alloc_init_and_set() function to
initialize crypto-engine and enable retry mechanism.

Set the maximum size for crypto-engine software queue based on
Job Ring size (JOBR_DEPTH) and a threshold (reserved for the
non-crypto-API requests that are not passed through crypto-engine).

The callback for do_batch_requests is NULL, since CAAM
doesn't support linked requests.
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Currently, by default crypto self-test failures only result in a
pr_warn() message and an "unknown" status in /proc/crypto.  Both of
these are easy to miss.  There is also an option to panic the kernel
when a test fails, but that can't be the default behavior.

A crypto self-test failure always indicates a kernel bug, however, and
there's already a standard way to report (recoverable) kernel bugs --
the WARN() family of macros.  WARNs are noisier and harder to miss, and
existing test systems already know to look for them in dmesg or via
/proc/sys/kernel/tainted.

Therefore, call WARN() when an algorithm fails its self-tests.
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When alg_test() is called from tcrypt.ko rather than from the algorithm
registration code, "driver" is actually the algorithm name, not the
driver name.  So it shouldn't be used in places where a driver name is
wanted, e.g. when reporting a test failure or when checking whether the
driver is the generic driver or not.

Fix this for the skcipher algorithm tests by getting the driver name
from the crypto_skcipher that actually got allocated.
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When alg_test() is called from tcrypt.ko rather than from the algorithm
registration code, "driver" is actually the algorithm name, not the
driver name.  So it shouldn't be used in places where a driver name is
wanted, e.g. when reporting a test failure or when checking whether the
driver is the generic driver or not.

Fix this for the AEAD algorithm tests by getting the driver name from
the crypto_aead that actually got allocated.
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When alg_test() is called from tcrypt.ko rather than from the algorithm
registration code, "driver" is actually the algorithm name, not the
driver name.  So it shouldn't be used in places where a driver name is
wanted, e.g. when reporting a test failure or when checking whether the
driver is the generic driver or not.

Fix this for the hash algorithm tests by getting the driver name from
the crypto_ahash or crypto_shash that actually got allocated.
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add crypto_aead_driver_name(), which is analogous to
crypto_skcipher_driver_name().
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

A break is not needed if it is preceded by a return
Signed-off-by: NTom Rix <trix@redhat.com>
Reviewed-by: NAlexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Unrolling the LOAD and BLEND loops improves performance by ~8% on x86_64
(tested on Broadwell Xeon) while not increasing code size too much.
Signed-off-by: NArvind Sankar <nivedita@alum.mit.edu>
Reviewed-by: NEric Biggers <ebiggers@google.com>
Acked-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This reduces code size substantially (on x86_64 with gcc-10 the size of
sha256_update() goes from 7593 bytes to 1952 bytes including the new
SHA256_K array), and on x86 is slightly faster than the full unroll
(tested on Broadwell Xeon).
Signed-off-by: NArvind Sankar <nivedita@alum.mit.edu>
Reviewed-by: NEric Biggers <ebiggers@google.com>
Acked-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The temporary W[] array is currently zeroed out once every call to
sha256_transform(), i.e. once every 64 bytes of input data. Moving it to
sha256_update() instead so that it is cleared only once per update can
save about 2-3% of the total time taken to compute the digest, with a
reasonable memset() implementation, and considerably more (~20%) with a
bad one (eg the x86 purgatory currently uses a memset() coded in C).
Signed-off-by: NArvind Sankar <nivedita@alum.mit.edu>
Reviewed-by: NEric Biggers <ebiggers@google.com>
Acked-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The assignments to clear a through h and t1/t2 are optimized out by the
compiler because they are unused after the assignments.

Clearing individual scalar variables is unlikely to be useful, as they
may have been assigned to registers, and even if stack spilling was
required, there may be compiler-generated temporaries that are
impossible to clear in any case.

So drop the clearing of a through h and t1/t2.
Signed-off-by: NArvind Sankar <nivedita@alum.mit.edu>
Reviewed-by: NEric Biggers <ebiggers@google.com>
Acked-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Without the barrier_data() inside memzero_explicit(), the compiler may
optimize away the state-clearing if it can tell that the state is not
used afterwards.
Signed-off-by: NArvind Sankar <nivedita@alum.mit.edu>
Acked-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Without the barrier_data() inside memzero_explicit(), the compiler may
optimize away the state-clearing if it can tell that the state is not
used afterwards. At least in lib/crypto/sha256.c:__sha256_final(), the
function can get inlined into sha256(), in which case the memset is
optimized away.
Signed-off-by: NArvind Sankar <nivedita@alum.mit.edu>
Reviewed-by: NEric Biggers <ebiggers@google.com>
Acked-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

pm_runtime_get_sync() will increment  pm usage counter even
when it returns an error code. We should call put operation
in error handling paths of omap_aes_hw_init.
Signed-off-by: NZhang Qilong <zhangqilong3@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This is an algorithm optimization. The reset operation when
setting the public key is repeated and redundant, so remove it.
At the same time, `sm2_ecc_os2ec()` is optimized to make the
function more simpler and more in line with the Linux code style.
Signed-off-by: NTianjia Zhang <tianjia.zhang@linux.alibaba.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch reduces the stack usage in sa2ul:

1. Move the exported sha state into sa_prepare_iopads so that it
can occupy the same space as the k_pad buffer.

2. Use one buffer for ipad/opad in sa_prepare_iopads.

3. Remove ipad/opad buffer from sa_set_sc_auth.

4. Use async skcipher fallback and remove on-stack request from
sa_cipher_run.
Reported-by: Nkernel test robot <lkp@intel.com>
Fixes: d2c8ac18 ("crypto: sa2ul - Add AEAD algorithm support")
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Clean up extra blank lines
Signed-off-by: NLongfang Liu <liulongfang@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

1. Remove unused member‘pending_reqs' in‘sec_qp_ctx' structure.
2. Remove unused member‘status'  in‘sec_dev' structure.
Signed-off-by: NLongfang Liu <liulongfang@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit 1d2c3279 ("crypto: x86/aes - drop scalar assembler
implementations") was meant to remove aes_glue.c, but it actually left
it as an unused one-line file.  Remove this unused file.

Cc: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Change type of ae_mask in adf_hw_device_data to allow for devices with
more than 16 Acceleration Engines (AEs).
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: NWojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: NFiona Trahe <fiona.trahe@intel.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Allow for crypto instances to be configured with symmetric crypto rings
that belong to a bank that is different from the one where asymmetric
crypto rings are located.

This is to allow for devices with banks made of a single ring pair.
In these, crypto instances will be composed of two separate banks.

Changed string literals are not exposed to the user space.
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: NWojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: NFiona Trahe <fiona.trahe@intel.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Refactor function qat_crypto_dev_config() to propagate errors to
the caller.
Suggested-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Refactor function qat_crypto_create_instances() to propagate errors to
the caller.
Suggested-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

If a key is not found in the internal key value storage, return -ENODATA
instead of -1 that is treated as -EPERM and may confuse.
Suggested-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

If the parameter type provided to adf_cfg_add_key_value_param()
is invalid, return -EINVAL instead of -1 that is treated as -EPERM and
may confuse.
Suggested-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Remove superfluous casts to void* in function qat_crypto_dev_config().
Suggested-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Call the function configure_iov_threads(), adf_enable_vf2pf_interrupts()
and adf_pf2vf_notify_restarting() only if present in the struct
adf_hw_device_data of the device.
This is to allow for QAT drivers that do not implement those functions.
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: NWojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: NMaksim Lukoshkov <maksim.lukoshkov@intel.com>
Reviewed-by: NFiona Trahe <fiona.trahe@intel.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>