Avoid excessive scheduling delays under a preemptible kernel by
yielding the NEON after every block of input.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Avoid excessive scheduling delays under a preemptible kernel by
yielding the NEON after every block of input.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Avoid excessive scheduling delays under a preemptible kernel by
yielding the NEON after every block of input.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Avoid excessive scheduling delays under a preemptible kernel by
yielding the NEON after every block of input.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Trivial fix to spelling mistake in module description text
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix a typo where size of RSA prime factor q is using the size of
prime factor p.

Cc: <stable@vger.kernel.org> # 4.13+
Fixes: 52e26d77 ("crypto: caam - add support for RSA key form 2")
Fixes: 4a651b12 ("crypto: caam - add support for RSA key form 3")
Reported-by: NDavid Binderman <dcb314@hotmail.com>
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In the quest to remove all stack VLA usage from the kernel[1], this
allocates the return code buffers before starting jiffie timers, rather
than using stack space for the array. Additionally cleans up some exit
paths and make sure that the num_mb module_param() is used only once
per execution to avoid possible races in the value changing.

[1] https://lkml.kernel.org/r/CA+55aFzCG-zNmZwX4A2FQpadafLfEzK6CC=qPXydAacU1RqZWA@mail.gmail.comSigned-off-by: NKees Cook <keescook@chromium.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add support for the SM4 symmetric cipher implemented using the special
SM4 instructions introduced in ARM architecture revision 8.2.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In preparation of adding support for the SIMD based arm64 implementation
of arm64, which requires a fallback to non-SIMD code when invoked in
certain contexts, expose the generic SM4 encrypt and decrypt routines
to other drivers.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: NGilad Ben-Yossef <gilad@benyossef.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When suspend is called after pm_runtime_suspend,
same callback is used and access to rng register is
freezing system. By calling the pm_runtime_force_suspend,
it first checks that runtime has been already done.
Signed-off-by: NLionel Debieve <lionel.debieve@st.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Define default state for stm32_rng driver. It will
be default selected with multi_v7_defconfig
Signed-off-by: NLionel Debieve <lionel.debieve@st.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix incorrect use of %pad as a printk format string for none dma_addr_t
variable.

Discovered via smatch.
Signed-off-by: NGilad Ben-Yossef <gilad@benyossef.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Enable CryptoCell support for hardware keys.

Hardware keys are regular AES keys loaded into CryptoCell internal memory
via firmware, often from secure boot ROM or hardware fuses at boot time.

As such, they can be used for enc/dec purposes like any other key but
cannot (read: extremely hard to) be extracted since since they are not
available anywhere in RAM during runtime.

The mechanism has some similarities to s390 secure keys although the keys
are not wrapped or sealed, but simply loaded offline. The interface was
therefore modeled based on the s390 secure keys support.
Signed-off-by: NGilad Ben-Yossef <gilad@benyossef.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes a crash that happens when testing rfc4543(gcm(aes))

Unable to handle kernel paging request for data at address 0xf59b3420
Faulting instruction address: 0xc0012994
Oops: Kernel access of bad area, sig: 11 [#1]
BE PowerPC 44x Platform
Modules linked in: tcrypt(+) crypto4xx [...]
CPU: 0 PID: 0 Comm: swapper Tainted: G           O      4.17.0-rc1+ #23
NIP:  c0012994 LR: d3077934 CTR: 06026d49
REGS: cfff7e30 TRAP: 0300   Tainted: G           O       (4.17.0-rc1+)
MSR:  00029000 <CE,EE,ME>  CR: 44744822  XER: 00000000
DEAR: f59b3420 ESR: 00000000
NIP [c0012994] __dma_sync+0x58/0x10c
LR [d3077934] crypto4xx_bh_tasklet_cb+0x188/0x3c8 [crypto4xx]

__dma_sync was fed the temporary _dst that crypto4xx_build_pd()
had in it's function stack. This clearly never worked.
This patch therefore overhauls the code from the original driver
and puts the temporary dst sg list into aead's request context.

Fixes: a0aae821 ("crypto: crypto4xx - prepare for AEAD support")
Signed-off-by: NChristian Lamparter <chunkeey@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

1020 bytes is the limit for associated data. Any more
and it will no longer fit into hash_crypto_offset anymore.

The hardware will not process aead requests with plaintext
that have less than AES_BLOCK_SIZE bytes. When decrypting
aead requests the authsize has to be taken in account as
well, as it is part of the cryptlen. Otherwise the hardware
will think it has been misconfigured and will return:

aead return err status = 0x98

For rtc4543(gcm(aes)), the hardware has a dedicated GMAC
mode as part of the hash function set.
Signed-off-by: NChristian Lamparter <chunkeey@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes cts(cbc(aes)) test when cbc-aes-ppc4xx is used.
alg: skcipher: Test 1 failed (invalid result) on encryption for cts(cbc-aes-ppc4xx)
00000000: 4b 10 75 fc 2f 14 1b 6a 27 35 37 33 d1 b7 70 05
00000010: 97
alg: skcipher: Failed to load transform for cts(cbc(aes)): -2

The CTS cipher mode expect the IV (req->iv) of skcipher_request
to contain the last ciphertext block after the {en,de}crypt
operation is complete.

Fix this issue for the AMCC Crypto4xx hardware engine.
The tcrypt test case for cts(cbc(aes)) is now correctly passed.

name         : cts(cbc(aes))
driver       : cts(cbc-aes-ppc4xx)
module       : cts
priority     : 300
refcnt       : 1
selftest     : passed
internal     : no
type         : skcipher
async        : yes
blocksize    : 16
min keysize  : 16
max keysize  : 32
ivsize       : 16
chunksize    : 16
walksize     : 16
Signed-off-by: NChristian Lamparter <chunkeey@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds support for the aes-ctr skcipher.

name         : ctr(aes)
driver       : ctr-aes-ppc4xx
module       : crypto4xx
priority     : 300
refcnt       : 1
selftest     : passed
internal     : no
type         : skcipher
async        : yes
blocksize    : 16
min keysize  : 16
max keysize  : 32
ivsize       : 16
chunksize    : 16
walksize     : 16

The hardware uses only the last 32-bits as the counter while the
kernel tests (aes_ctr_enc_tv_template[4] for example) expect that
the whole IV is a counter. To make this work, the driver will
fallback if the counter is going to overlow.

The aead's crypto4xx_setup_fallback() function is renamed to
crypto4xx_aead_setup_fallback.
Signed-off-by: NChristian Lamparter <chunkeey@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes some of the -Wvla warnings.

crypto4xx_alg.c:83:19: warning: Variable length array is used.
crypto4xx_alg.c:273:56: warning: Variable length array is used.
crypto4xx_alg.c:380:32: warning: Variable length array is used.
Signed-off-by: NChristian Lamparter <chunkeey@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The ablkcipher APIs have been effectively deprecated since [1].
This patch converts the crypto4xx driver to the new skcipher APIs.

[1] <https://www.spinics.net/lists/linux-crypto/msg18133.html>
Signed-off-by: NChristian Lamparter <chunkeey@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch provides a cheap 2MiB/s+ (~ 6%) performance
improvement over the current code. This is because the
compiler can now optimize several endian swap memcpy.
Signed-off-by: NChristian Lamparter <chunkeey@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes the clang warning of extraneous parentheses, with the
following coccinelle script.

@@
identifier i;
constant c;
expression e;
@@
(
!((e))
|
-((
\(i == c\|i != c\|i <= c\|i < c\|i >= c\|i > c\)
-))
)
Signed-off-by: NVarsha Rao <rvarsha016@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

We should get drvdata from struct device directly. Going via
platform_device is an unneeded step back and forth.
Signed-off-by: NWolfram Sang <wsa+renesas@sang-engineering.com>
Reviewed-by: NKrzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fixes: 8043bb1a ("crypto: omap-sham - convert driver logic to use sgs for data xmit")

The memory pages freed in omap_sham_finish_req() were less than those
allocated in omap_sham_copy_sgs().

Cc: stable@vger.kernel.org
Signed-off-by: NBin Liu <b-liu@ti.com>
Acked-by: NTero Kristo <t-kristo@ti.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Remove dependencies on HAS_DMA where a Kconfig symbol depends on another
symbol that implies HAS_DMA, and, optionally, on "|| COMPILE_TEST".
In most cases this other symbol is an architecture or platform specific
symbol, or PCI.

Generic symbols and drivers without platform dependencies keep their
dependencies on HAS_DMA, to prevent compiling subsystems or drivers that
cannot work anyway.

This simplifies the dependencies, and allows to improve compile-testing.
Signed-off-by: NGeert Uytterhoeven <geert@linux-m68k.org>
Reviewed-by: NMark Brown <broonie@kernel.org>
Acked-by: NRobin Murphy <robin.murphy@arm.com>
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use kmemdup() rather than duplicating its implementation.

By usign kmemdup() we can also get rid of the 'val' variable.

Detected with Coccinelle script.
Signed-off-by: NFabio Estevam <fabio.estevam@nxp.com>
Reviewed-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Sometimes the provided RSA input buffer provided is not stripped
of leading zeros. This could cause its size to be bigger than that
of the modulus, making the HW complain:

caam_jr 2142000.jr1: 40000789: DECO: desc idx 7:
Protocol Size Error - A protocol has seen an error in size. When
running RSA, pdb size N < (size of F) when no formatting is used; or
pdb size N < (F + 11) when formatting is used.

Fix the problem by stripping off the leading zero from input data
before feeding it to the CAAM accelerator.

Fixes: 8c419778 ("crypto: caam - add support for RSA algorithm")
Cc: <stable@vger.kernel.org> # 4.8+
Reported-by: NMartin Townsend <mtownsend1973@gmail.com>
Link: https://lkml.kernel.org/r/CABatt_ytYORYKtApcB4izhNanEKkGFi9XAQMjHi_n-8YWoCRiw@mail.gmail.comSigned-off-by: NHoria Geantă <horia.geanta@nxp.com>
Tested-by: NFabio Estevam <fabio.estevam@nxp.com>
Reviewed-by: NTudor Ambarus <tudor.ambarus@microchip.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

New Centaur CPU(Family > 6) supprt Random Number Generator, but can't
support MSR_VIA_RNG. Just like VIA Nano.
Signed-off-by: NDavid Wang <davidwang@zhaoxin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

There is no need to assign an error value to 'ret' prior
to calling mpi_read_raw_from_sgl() because in the case
of error the 'ret' variable will be assigned to the error
code inside the if block.

In the case of non failure, 'ret' will be overwritten
immediately after, so remove the unneeded assignment.
Signed-off-by: NFabio Estevam <fabio.estevam@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The following error is triggered by the ThunderX ZIP driver
if the testmanager is enabled:

[  199.069437] ThunderX-ZIP 0000:03:00.0: Found ZIP device 0 177d:a01a on Node 0
[  199.073573] alg: comp: Compression test 1 failed for deflate-generic: output len = 37

The reason for this error is the verification of the compression
results. Verifying the compression result only works if all
algorithm parameters are identical, in this case to the software
implementation.

Different compression engines like the ThunderX ZIP coprocessor
might yield different compression results by tuning the
algorithm parameters. In our case the compressed result is
shorter than the test vector.

We should not forbid different compression results but only
check that compression -> decompression yields the same
result. This is done already in the acomp test. Do something
similar for test_comp().
Signed-off-by: NMahipal Challa <mchalla@cavium.com>
Signed-off-by: NBalakrishna Bhamidipati <bbhamidipati@cavium.com>
[jglauber@cavium.com: removed unrelated printk changes, rewrote commit msg,
 fixed whitespace and unneeded initialization]
Signed-off-by: NJan Glauber <jglauber@cavium.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The 'era' information can be retrieved from CAAM registers, so
introduce a caam_get_era_from_hw() function that gets it via register
reads in case the 'fsl,sec-era' property is not passed in the device
tree.

This function is based on the U-Boot implementation from
drivers/crypto/fsl/sec.c
Signed-off-by: NFabio Estevam <fabio.estevam@nxp.com>
Reviewed-by: NHoria Geantă <horia.geanta@nxp.com>
Tested-by: NBreno Lima <breno.lima@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

caam_get_era() is only used locally, so do not export this function
and make it static instead.
Signed-off-by: NFabio Estevam <fabio.estevam@nxp.com>
Reviewed-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Switch to raw_smp_processor_id() to prevent a number of
warnings from kernel debugging. We do not care about
preemption here, as the CPU number is only used as a
poor mans load balancing or device selection. If preemption
happens during a compress/decompress operation a small performance
hit will occur but everything will continue to work, so just
ignore it.
Signed-off-by: NJan Glauber <jglauber@cavium.com>
Reviewed-by: NRobert Richter <rrichter@cavium.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The pending request counter was read from the wrong register. While
at it, there is no need to use an atomic for it as it is only read
localy in a loop.
Signed-off-by: NJan Glauber <jglauber@cavium.com>
Reviewed-by: NRobert Richter <rrichter@cavium.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Avoid two potential divisions by zero when calculating average
values for the zip statistics.
Signed-off-by: NJan Glauber <jglauber@cavium.com>
Reviewed-by: NRobert Richter <rrichter@cavium.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

After issuing a request an endless loop was used to read the
completion state from memory which is asynchronously updated
by the ZIP coprocessor.

Add an upper bound to the retry attempts to prevent a CPU getting stuck
forever in case of an error. Additionally, add a read memory barrier
and a small delay between the reading attempts.
Signed-off-by: NJan Glauber <jglauber@cavium.com>
Reviewed-by: NRobert Richter <rrichter@cavium.com>
Cc: stable <stable@vger.kernel.org> # 4.14
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Enabling virtual mapped kernel stacks breaks the thunderx_zip
driver. On compression or decompression the executing CPU hangs
in an endless loop. The reason for this is the usage of __pa
by the driver which does no longer work for an address that is
not part of the 1:1 mapping.

The zip driver allocates a result struct on the stack and needs
to tell the hardware the physical address within this struct
that is used to signal the completion of the request.

As the hardware gets the wrong address after the broken __pa
conversion it writes to an arbitrary address. The zip driver then
waits forever for the completion byte to contain a non-zero value.

Allocating the result struct from 1:1 mapped memory resolves this
bug.
Signed-off-by: NJan Glauber <jglauber@cavium.com>
Reviewed-by: NRobert Richter <rrichter@cavium.com>
Cc: stable <stable@vger.kernel.org> # 4.14
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

We avoid various VLAs[1] by using constant expressions for block size
and alignment mask.

[1] http://lkml.kernel.org/r/CA+55aFzCG-zNmZwX4A2FQpadafLfEzK6CC=qPXydAacU1RqZWA@mail.gmail.comSigned-off-by: NSalvatore Mesoraca <s.mesoraca16@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In preparation for the removal of VLAs[1] from crypto code.
We create 2 new compile-time constants: all ciphers implemented
in Linux have a block size less than or equal to 16 bytes and
the most demanding hw require 16 bytes alignment for the block
buffer.
We also enforce these limits in crypto_check_alg when a new
cipher is registered.

[1] http://lkml.kernel.org/r/CA+55aFzCG-zNmZwX4A2FQpadafLfEzK6CC=qPXydAacU1RqZWA@mail.gmail.comSigned-off-by: NSalvatore Mesoraca <s.mesoraca16@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

There is a double assignment to cdev->ports, the first is redundant
as it is over-written so remove it.

Detected by CoverityScan, CID#1467432 ("Unused value")
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The structure crypto_info contains fields that are not initialized and
only .version is set.  The copy_to_user call is hence leaking information
from the stack to userspace which must be avoided. Fix this by zero'ing
all the unused fields.

Detected by CoverityScan, CID#1467421 ("Uninitialized scalar variable")

Fixes: a0894394 ("crypto: chtls - Register chtls with net tls")
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>