This patch includes a missing dependency (CRYPTO_AES) which may
lead to an "undefined reference to `aes_expandkey'" linking error.

Fixes: 5106dfea ("crypto: qat - add AES-XTS support for QAT GEN4 devices")
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NMarco Chiappero <marco.chiappero@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add dependency for CRYPTO_DEV_KEEMBAY_OCS_AES_SM4 on HAS_IOMEM to
prevent build failures.

Fixes: 88574332 ("crypto: keembay - Add support for Keem Bay OCS AES/SM4")
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NDaniele Alessandrelli <daniele.alessandrelli@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The Intel Keem Bay Offload and Crypto Subsystem (OCS) is only present on
Intel Keem Bay SoCs.  Hence add a dependency on ARCH_KEEMBAY, to prevent
asking the user about this driver when configuring a kernel without
Intel Keem Bay platform support.

While at it, fix a misspelling of "cipher".

Fixes: 88574332 ("crypto: keembay - Add support for Keem Bay OCS AES/SM4")
Signed-off-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Acked-by: NDaniele Alessandrelli <daniele.alessandrelli@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add logic to detect device capabilities in qat_4xxx driver.

Read fuses and build the device capabilities mask. This will enable
services and handling specific to QAT 4xxx devices.
Co-developed-by: NTomaszx Kowalik <tomaszx.kowalik@intel.com>
Signed-off-by: NTomaszx Kowalik <tomaszx.kowalik@intel.com>
Signed-off-by: NMarco Chiappero <marco.chiappero@intel.com>
Reviewed-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add handling of AES-XTS specific to QAT GEN4 devices.
Co-developed-by: NTomaszx Kowalik <tomaszx.kowalik@intel.com>
Signed-off-by: NTomaszx Kowalik <tomaszx.kowalik@intel.com>
Signed-off-by: NMarco Chiappero <marco.chiappero@intel.com>
Reviewed-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add support for AES-CTR for QAT GEN4 devices.
Also, introduce the capability ICP_ACCEL_CAPABILITIES_AES_V2 and the
helper macro HW_CAP_AES_V2, which allow to distinguish between
different HW generations.
Co-developed-by: NTomasz Kowalik <tomaszx.kowalik@intel.com>
Signed-off-by: NTomasz Kowalik <tomaszx.kowalik@intel.com>
Co-developed-by: NMateusz Polrola <mateuszx.potrola@intel.com>
Signed-off-by: NMateusz Polrola <mateuszx.potrola@intel.com>
Signed-off-by: NMarco Chiappero <marco.chiappero@intel.com>
Reviewed-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The bitreverse helper is almost always built into the kernel,
but in a rare randconfig build it is possible to hit a case
in which it is a loadable module while the atmel-i2c driver
is built-in:

arm-linux-gnueabi-ld: drivers/crypto/atmel-i2c.o: in function `atmel_i2c_checksum':
atmel-i2c.c:(.text+0xa0): undefined reference to `byte_rev_table'

Add one more 'select' statement to prevent this.

Fixes: 11105693 ("crypto: atmel-ecc - introduce Microchip / Atmel ECC driver")
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

a set of atomic_inc_return() looks more neater
Signed-off-by: NYejune Deng <yejune.deng@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add support for the AES/SM4 crypto engine included in the Offload and
Crypto Subsystem (OCS) of the Intel Keem Bay SoC, thus enabling
hardware-acceleration for the following transformations:

- ecb(aes), cbc(aes), ctr(aes), cts(cbc(aes)), gcm(aes) and cbc(aes);
  supported for 128-bit and 256-bit keys.

- ecb(sm4), cbc(sm4), ctr(sm4), cts(cbc(sm4)), gcm(sm4) and cbc(sm4);
  supported for 128-bit keys.

The driver passes crypto manager self-tests, including the extra tests
(CRYPTO_MANAGER_EXTRA_TESTS=y).
Signed-off-by: NMike Healy <mikex.healy@intel.com>
Co-developed-by: NDaniele Alessandrelli <daniele.alessandrelli@intel.com>
Signed-off-by: NDaniele Alessandrelli <daniele.alessandrelli@intel.com>
Acked-by: NMark Gross <mgross@linux.intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add device-tree bindings for Intel Keem Bay Offload and Crypto Subsystem
(OCS) AES crypto driver.
Signed-off-by: NDaniele Alessandrelli <daniele.alessandrelli@intel.com>
Acked-by: NMark Gross <mgross@linux.intel.com>
Reviewed-by: NRob Herring <robh@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Geert reports that builds where CONFIG_CRYPTO_AEGIS128_SIMD is not set
may still emit references to crypto_aegis128_update_simd(), which
cannot be satisfied and therefore break the build. These references
only exist in functions that can be optimized away, but apparently,
the compiler is not always able to prove this.

So add some explicit checks for CONFIG_CRYPTO_AEGIS128_SIMD to help the
compiler figure this out.
Tested-by: NGeert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The macro use will already have a semicolon.
Signed-off-by: NTom Rix <trix@redhat.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

CMP $0,%reg can't set overflow flag, so we can use shorter TEST %reg,%reg
instruction when only zero and sign flags are checked (E,L,LE,G,GE conditions).
Signed-off-by: NUros Bizjak <ubizjak@gmail.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

CMP $0,%reg can't set overflow flag, so we can use shorter TEST %reg,%reg
instruction when only zero and sign flags are checked (E,L,LE,G,GE conditions).
Signed-off-by: NUros Bizjak <ubizjak@gmail.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

CMP $0,%reg can't set overflow flag, so we can use shorter TEST %reg,%reg
instruction when only zero and sign flags are checked (E,L,LE,G,GE conditions).
Signed-off-by: NUros Bizjak <ubizjak@gmail.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes a few sparse warnings that were missed in the
last round.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds a dependency for KEYSTONE on HAS_IOMEM and OF to
prevent COMPILE_TEST build failures.
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes a missing prototype warning on blake2s_selftest.
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

ARM Cortex-A57 and Cortex-A72 cores running in 32-bit mode are affected
by silicon errata #1742098 and #1655431, respectively, where the second
instruction of a AES instruction pair may execute twice if an interrupt
is taken right after the first instruction consumes an input register of
which a single 32-bit lane has been updated the last time it was modified.

This is not such a rare occurrence as it may seem: in counter mode, only
the least significant 32-bit word is incremented in the absence of a
carry, which makes our counter mode implementation susceptible to these
errata.

So let's shuffle the counter assignments around a bit so that the most
recent updates when the AES instruction pair executes are 128-bit wide.

[0] ARM-EPM-049219 v23 Cortex-A57 MPCore Software Developers Errata Notice
[1] ARM-EPM-012079 v11.0 Cortex-A72 MPCore Software Developers Errata Notice

Cc: <stable@vger.kernel.org> # v5.4+
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

ecdh_set_secret() casts a void* pointer to a const u64* in order to
feed it into ecc_is_key_valid(). This is not generally permitted by
the C standard, and leads to actual misalignment faults on ARMv6
cores. In some cases, these are fixed up in software, but this still
leads to performance hits that are entirely avoidable.

So let's copy the key into the ctx buffer first, which we will do
anyway in the common case, and which guarantees correct alignment.

Cc: <stable@vger.kernel.org>
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Rework the setting of DMA cache parameters, program more appropriate
values and explicitly set sharability domain.
Signed-off-by: NGilad Ben-Yossef <gilad@benyossef.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

'pci_set_dma_mask()' + 'pci_set_consistent_dma_mask()' can be replaced by
an equivalent 'dma_set_mask_and_coherent()' which is much less verbose.
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

'pci_set_dma_mask()' + 'pci_set_consistent_dma_mask()' can be replaced by
an equivalent 'dma_set_mask_and_coherent()' which is much less verbose.
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

'pci_set_dma_mask()' + 'pci_set_consistent_dma_mask()' can be replaced by
an equivalent 'dma_set_mask_and_coherent()' which is much less verbose.
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In preparation to enable -Wimplicit-fallthrough for Clang, fix multiple
warnings by explicitly adding multiple break statements instead of
letting the code fall through to the next case.

Link: https://github.com/KSPP/linux/issues/115Signed-off-by: NGustavo A. R. Silva <gustavoars@kernel.org>
Acked-by: NGilad Ben-Yossef <gilad@benyossef.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

WireGuard and IPsec both typically operate on input blocks that are
~1420 bytes in size, given the default Ethernet MTU of 1500 bytes and
the overhead of the VPN metadata.

Many aead and sckipher implementations are optimized for power-of-2
block sizes, and whether they perform well when operating on 1420
byte blocks cannot be easily extrapolated from the performance on
power-of-2 block size. So let's add 1420 bytes explicitly, and round
it up to the next blocksize multiple of the algo in question if it
does not support 1420 byte blocks.
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When working on crypto algorithms, being able to run tcrypt quickly
without booting an entire Linux installation can be very useful. For
instance, QEMU/kvm can be used to boot a kernel from the command line,
and having tcrypt.ko builtin would allow tcrypt to be executed to run
benchmarks, or to run tests for algorithms that need to be instantiated
from templates, without the need to make it past the point where the
rootfs is mounted.

So let's relax the requirement that tcrypt can only be built as a module
when CONFIG_EXPERT is enabled.
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit c4741b23 ("crypto: run initcalls for generic implementations
earlier") converted tcrypt.ko's module_init() to subsys_initcall(), but
this was unintentional: tcrypt.ko currently cannot be built into the core
kernel, and so the subsys_initcall() gets converted into module_init()
under the hood. Given that tcrypt.ko does not implement a generic version
of a crypto algorithm that has to be available early during boot, there
is no point in running the tcrypt init code earlier than implied by
module_init().

However, for crypto development purposes, we will lift the restriction
that tcrypt.ko must be built as a module, and when builtin, it makes sense
for tcrypt.ko (which does its work inside the module init function) to run
as late as possible. So let's switch to late_initcall() instead.
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Reviewed-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Move HiSilicon TRNG V2 driver into 'drivers/crypto/hisilicon/trng'
with some updating on 'MAINTAINERS'.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds support for pseudo random number generator(PRNG)
in Crypto subsystem.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Move existing char/hw_random/hisi-trng-v2.c to crypto/hisilicon/trng.c.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Driver of HiSilicon true random number generator(TRNG)
is removed from 'drivers/char/hw_random'.

Both 'Kunpeng 920' and 'Kunpeng 930' chips have TRNG,
however, PRNG is only supported by 'Kunpeng 930'.
So, this driver is moved to 'drivers/crypto/hisilicon/trng/'
in the next to enable the two's TRNG better.
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Reviewed-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes a coulpe of sparse endianness warnings.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes a sparse endianness warning in sha256-spe.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes a number of endianness warnings in the mips/octeon
code.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

 Condition !A || A && B is equivalent to !A || B.

Generated by: scripts/coccinelle/misc/excluded_middle.cocci

Fixes: b76f0ea0 ("coccinelle: misc: add excluded_middle.cocci script")
CC: Denis Efremov <efremov@linux.com>
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NJulia Lawall <julia.lawall@inria.fr>
Signed-off-by: NGiovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Partial hash was being copied into the final result buffer without the
entire message block processed. Depending on how the end user processes
this result buffer, errors vary from result buffer corruption to result
buffer poisoing. Fix this issue by ensuring that only the final hash value
is copied into the result buffer.
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>
Signed-off-by: NThara Gopinath <thara.gopinath@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add support Qualcomm Crypto Engine accelerated encryption and
authentication algorithms on sdm845.
Reviewed-by: NBjorn Andersson <bjorn.andersson@linaro.org>
Signed-off-by: NThara Gopinath <thara.gopinath@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Wiring the SIMD code into the generic driver has the unfortunate side
effect that the tcrypt testing code cannot distinguish them, and will
therefore not use the latter to fuzz test the former, as it does for
other algorithms.

So let's refactor the code a bit so we can register two implementations:
aegis128-generic and aegis128-simd.
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Reviewed-by: NOndrej Mosnacek <omosnacek@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Instead of calculating the tag and returning it to the caller on
decryption, use a SIMD compare and min across vector to perform
the comparison. This is slightly more efficient, and removes the
need on the caller's part to wipe the tag from memory if the
decryption failed.

While at it, switch to unsigned int when passing cryptlen and
assoclen - we don't support input sizes where it matters anyway.
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Reviewed-by: NOndrej Mosnacek <omosnacek@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>