This integrates the accelerated MIPS 32r2 implementation of ChaCha
into both the API and library interfaces of the kernel crypto stack.

The significance of this is that, in addition to becoming available
as an accelerated library implementation, it can also be used by
existing crypto API code such as Adiantum (for block encryption on
ultra low performance cores) or IPsec using chacha20poly1305. These
are use cases that have already opted into using the abstract crypto
API. In order to support Adiantum, the core assembler routine has
been adapted to take the round count as a function argument rather
than hardcoding it to 20.
Co-developed-by: NRené van Dorst <opensource@vdorst.com>
Signed-off-by: NRené van Dorst <opensource@vdorst.com>
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This imports the accelerated MIPS 32r2 ChaCha20 implementation from the
Zinc patch set.
Co-developed-by: NRené van Dorst <opensource@vdorst.com>
Signed-off-by: NRené van Dorst <opensource@vdorst.com>
Signed-off-by: NJason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Expose the accelerated NEON ChaCha routine directly as a symbol
export so that users of the ChaCha library API can use it directly.

Given that calls into the library API will always go through the
routines in this module if it is enabled, switch to static keys
to select the optimal implementation available (which may be none
at all, in which case we defer to the generic implementation for
all invocations).
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Instead of falling back to the generic ChaCha skcipher driver for
non-SIMD cases, use a fast scalar implementation for ARM authored
by Eric Biggers. This removes the module dependency on chacha-generic
altogether, which also simplifies things when we expose the ChaCha
library interface from this module.
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Expose the accelerated NEON ChaCha routine directly as a symbol
export so that users of the ChaCha library API can use it directly.

Given that calls into the library API will always go through the
routines in this module if it is enabled, switch to static keys
to select the optimal implementation available (which may be none
at all, in which case we defer to the generic implementation for
all invocations).
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Depend on the generic ChaCha library routines instead of pulling in the
generic ChaCha skcipher driver, which is more than we need, and makes
managing the dependencies between the generic library, generic driver,
accelerated library and driver more complicated.

While at it, drop the logic to prefer the scalar code on short inputs.
Turning the NEON on and off is cheap these days, and one major use case
for ChaCha20 is ChaCha20-Poly1305, which is guaranteed to hit the scalar
path upon every invocation  (when doing the Poly1305 nonce generation)
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Wire the existing x86 SIMD ChaCha code into the new ChaCha library
interface, so that users of the library interface will get the
accelerated version when available.

Given that calls into the library API will always go through the
routines in this module if it is enabled, switch to static keys
to select the optimal implementation available (which may be none
at all, in which case we defer to the generic implementation for
all invocations).
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In preparation of extending the x86 ChaCha driver to also expose the ChaCha
library interface, drop the dependency on the chacha_generic crypto driver
as a non-SIMD fallback, and depend on the generic ChaCha library directly.
This way, we only pull in the code we actually need, without registering
a set of ChaCha skciphers that we will never use.

Since turning the FPU on and off is cheap these days, simplify the SIMD
routine by dropping the per-page yield, which makes for a cleaner switch
to the library API as well. This also allows use to invoke the skcipher
walk routines in non-atomic mode.
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Currently, our generic ChaCha implementation consists of a permute
function in lib/chacha.c that operates on the 64-byte ChaCha state
directly [and which is always included into the core kernel since it
is used by the /dev/random driver], and the crypto API plumbing to
expose it as a skcipher.

In order to support in-kernel users that need the ChaCha streamcipher
but have no need [or tolerance] for going through the abstractions of
the crypto API, let's expose the streamcipher bits via a library API
as well, in a way that permits the implementation to be superseded by
an architecture specific one if provided.

So move the streamcipher code into a separate module in lib/crypto,
and expose the init() and crypt() routines to users of the library.
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In preparation of introducing a set of crypto library interfaces, tidy
up the Makefile and split off the Kconfig symbols into a separate file.
Signed-off-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

If aead is built as a module along with cryptomgr, it creates a
dependency loop due to the dependency chain aead => crypto_null =>
cryptomgr => aead.

This is due to the presence of the AEAD geniv code.  This code is
not really part of the AEAD API but simply support code for IV
generators such as seqiv.  This patch moves the geniv code into
its own module thus breaking the dependency loop.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The crypto API requires cryptomgr to be present for probing to work
so we need a softdep to ensure that cryptomgr is added to the
initramfs.

This was usually not a problem because until very recently it was
not practical to build crypto API as module but with the recent
work to eliminate direct AES users this is now possible.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The linux-amlogic mailing list need to be in copy of all patch for the amlogic crypto.
Signed-off-by: NCorentin Labbe <clabbe@baylibre.com>
Acked-by: NNeil Armstrong <narmstrong@baylibre.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes two resources leak that occur on error path.
Reported-by: Ncoverity-bot <keescook+coverity-bot@chromium.org>
Addresses-Coverity-ID: 1487403 ("RESOURCE_LEAK")
Addresses-Coverity-ID: 1487401 ("Resource leaks")
Fixes: 48fe583f ("crypto: amlogic - Add crypto accelerator for amlogic GXL")
Signed-off-by: NCorentin Labbe <clabbe@baylibre.com>
Reviewed-by: NKees Cook <keescook@chromium.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fixed 2 copy-paste mistakes in the commit mentioned below that caused
authenc w/ (3)DES to consistently fail on Macchiatobin (but strangely
work fine on x86+FPGA??).
Now fully tested on both platforms.

Fixes: 13a1bb93 ("crypto: inside-secure - Fixed warnings...")
Signed-off-by: NPascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When calling debugfs functions, there is no need to ever check the
return value.  The function can work or not, but the code logic should
never do something different based on this.

Cc: Zhou Wang <wangzhou1@hisilicon.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Currently the VF can be enabled only through sysfs interface
after module loaded, but this also needs to be done when the
module loaded in some scenarios.

This patch adds module param vfs_num, adds hisi_zip_sriov_enable()
in probe, and also adjusts the position of probe.
Signed-off-by: NHao Fang <fanghao11@huawei.com>
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fixed mask used for CFSIZE and RFSIZE fields of HIA_OPTIONS register,
these were all 1 bit too wide. Which caused the probing of a standard
EIP97 to actually hang due to assume way too large descriptor FIFO's.
Signed-off-by: NPascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Current polling timeout is 25 us. The hardware is currently configured to
harvest the entropy for 81920 us. This leads to timeouts even during
blocking read (wait=1).

Log snippet:
[    5.727589] [<c040ffcc>] (ks_sa_rng_probe) from [<c04181e8>] (platform_drv_probe+0x58/0xb4)
...
[    5.727805] hwrng: no data available
...
[   13.157016] random: systemd: uninitialized urandom read (16 bytes read)
[   13.157033] systemd[1]: Initializing machine ID from random generator.
...
[   15.848770] random: fast init done
...
[   15.848807] random: crng init done

After the patch:
[    6.223534] random: systemd: uninitialized urandom read (16 bytes read)
[    6.223551] systemd[1]: Initializing machine ID from random generator.
...
[    6.876075] random: fast init done
...
[    6.954200] random: systemd: uninitialized urandom read (16 bytes read)
[    6.955244] random: systemd: uninitialized urandom read (16 bytes read)
...
[    7.121948] random: crng init done
Signed-off-by: NAlexander Sverdlin <alexander.sverdlin@nokia.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix the warning below.
./crypto/tgr192.c:558:43-44: Unneeded semicolon
./crypto/tgr192.c:586:44-45: Unneeded semicolon

Fixes: f63fbd3d ("crypto: tgr192 - Switch to shash")
Signed-off-by: NTian Tao <tiantao6@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

There are spelling mistakes in dev_warn and dev_err messages. Fix these.
Change "recommandation" to "recommendation" and "tryed" to "tried".
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Acked-by: NCorentin Labbe <clabbe.montjoie@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add platform support for the new IP found on sam9x60 SoC. For this
version, if the peripheral clk is above 100MHz, the HALFR bit must be
set. This bit is available only if the IP can generate a random number
every 168 cycles (instead of 84).
Signed-off-by: NCodrin Ciubotariu <codrin.ciubotariu@microchip.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add compatible for new IP found on sam9x60 SoC.
Signed-off-by: NCodrin Ciubotariu <codrin.ciubotariu@microchip.com>
Acked-by: NRob Herring <robh@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The following error is raised when CONFIG_CRYPTO_DEV_ATMEL_AES=y and
CONFIG_CRYPTO_DEV_ATMEL_AUTHENC=m:
drivers/crypto/atmel-aes.o: In function `atmel_aes_authenc_setkey':
atmel-aes.c:(.text+0x9bc): undefined reference to `crypto_authenc_extractkeys'
Makefile:1094: recipe for target 'vmlinux' failed

Fix it by moving the selection of CRYPTO_AUTHENC under
config CRYPTO_DEV_ATMEL_AES.

Fixes: 89a82ef8 ("crypto: atmel-authenc - add support to...")
Signed-off-by: NTudor Ambarus <tudor.ambarus@microchip.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

safexcel_remove misses disabling priv->reg_clk like what is done when
probe fails.
Add the missed call to fix it.
Signed-off-by: NChuhong Yuan <hslester96@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Replace #ifdef CONFIG_NUMA with IS_ENABLED(CONFIG_NUMA) to fix kbuild error.

Fixes: 700f7d0d ("crypto: hisilicon - fix to return...")
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Reported-by: Nkbuild test robot <lkp@intel.com>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use kmemdup rather than duplicating its implementation

Generated by: scripts/coccinelle/api/memdup.cocci

Fixes: f08fcced ("crypto: allwinner - Add sun8i-ss cryptographic offloader")
CC: Corentin Labbe <clabbe.montjoie@gmail.com>
Signed-off-by: Nkbuild test robot <lkp@intel.com>
Signed-off-by: NJulia Lawall <julia.lawall@lip6.fr>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Generated by: scripts/coccinelle/misc/semicolon.cocci

Fixes: f08fcced ("crypto: allwinner - Add sun8i-ss cryptographic offloader")
CC: Corentin Labbe <clabbe.montjoie@gmail.com>
Signed-off-by: Nkbuild test robot <lkp@intel.com>
Signed-off-by: NJulia Lawall <julia.lawall@lip6.fr>
Acked-by: NCorentin Labbe <clabbe.montjoie@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes a crash that can happen during probe
when the available dma memory is not enough (this can
happen if the crypto4xx is built as a module).

The descriptor window mapping would end up being free'd
twice, once in crypto4xx_build_pdr() and the second time
in crypto4xx_destroy_sdr().

Fixes: 5d59ad6e ("crypto: crypto4xx - fix crypto4xx_build_pdr, crypto4xx_build_sdr leak")
Cc: <stable@vger.kernel.org>
Signed-off-by: NChristian Lamparter <chunkeey@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Here adds maintainer information for HiSilicon TRNG V2 driver.
Signed-off-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This series adds HiSilicon true random number generator(TRNG)
driver in hw_random subsystem.
Signed-off-by: NZaibo Xu <xuzaibo@huawei.com>
Signed-off-by: NWeili Qian <qianweili@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Currently when the call to crypto_engine_alloc_init fails the error
return path returns an uninitialized value in the variable err. Fix
this by setting err to -ENOMEM.

Addresses-Coverity: ("Uninitialized scalar variable")
Fixes: 48fe583f ("crypto: amlogic - Add crypto accelerator for amlogic GXL")
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Acked-by: NCorentin Labbe <clabbe@baylibre.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use kmemdup rather than duplicating its implementation
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Acked-by: NCorentin Labbe <clabbe@baylibre.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes the remaining sparse endianness warnings.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

As it is if CONFIG_CRYPTO_DEV_ATMEL_AUTHENC is set to m it is in
effect disabled.  This patch fixes it by using IS_ENABLED instead
of ifdef.

Fixes: 89a82ef8 ("crypto: atmel-authenc - add support to...")
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
Reviewed-by: NTudor Ambarus <tudor.ambarus@microchip.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch restores the CRYPTO_AES dependency. This is
necessary since some of the crypto4xx driver provided
modes need functioning software fallbacks for
AES-CTR/CCM and GCM.

Fixes: da3e7a97 ("crypto: amcc - switch to AES library for GCM key derivation")
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NChristian Lamparter <chunkeey@gmail.com>
Acked-by: NArd Biesheuvel <ardb@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Currently find_zip_device() finds zip device which has the min NUMA
distance with current CPU.

This patch modifies find_zip_device to return sub-optimal device when best
device has no qps. This patch sorts all devices by NUMA distance, then
finds the best zip device which has free qp.
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NShukun Tan <tanshukun1@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use sgl API to get sgl dma addr and len, this will help to avoid compile
error in some platforms. So NEED_SG_DMA_LENGTH can be removed here, which
can only be selected by arch code.
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Suggested-by: NHerbert Xu <herbert@gondor.apana.org.au>
Acked-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Now that the blkcipher algorithm type has been removed in favor of
skcipher, rename the crypto_blkcipher kernel module to crypto_skcipher,
and rename the config options accordingly:

	CONFIG_CRYPTO_BLKCIPHER => CONFIG_CRYPTO_SKCIPHER
	CONFIG_CRYPTO_BLKCIPHER2 => CONFIG_CRYPTO_SKCIPHER2
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>