This patch fixes a buffer overflow error returning -EINVAL for AEAD
decrypt operations by NOT appending the (already verified) ICV to
the output packet (which is not expected by the API anyway).
With this fix, all testmgr AEAD (extra) tests now pass.
Signed-off-by: NPascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fixed issues with the skcipher and AEAD scatter/gather list to engine
descriptor conversion code which caused either too much or too little
buffer space to be provided to the hardware. This caused errors with the
testmgr extra tests, either kernel panics (on x86-EIP197-FPGA) or engine
descriptor errors 0x1, 0x8 or 0x9 (on Macchiatobin e.g. Marvell A8K).
With this patch in place, all skcipher and AEAD (extra) tests pass.
Signed-off-by: NPascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes corruption issues with the skcipher output IV
witnessed on x86+EIP197-FPGA (devboard). The original fix, commit
57660b11 ("crypto: inside-secure - implement IV retrieval"),
attempted to write out the result IV through the context record.
However, this is not a reliable mechanism as there is no way of
knowing the hardware context update actually arrived in memory, so
it is possible to read the old contents instead of the updated IV.
(and indeed, this failed for the x86/FPGA case)

The alternative approach used here recognises the fact that the
result IV for CBC is actually the last cipher block, which is the last
input block in case of decryption and the last output block in case
of encryption. So the result IV is taken from the input data buffer
respectively the output data buffer instead, which *is* reliable.
Signed-off-by: NPascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The driver incorrectly advertised the IV size for DES and 3DES ECB
mode as being the DES blocksize of 8. This is incorrect as ECB mode
does not need any IV.
Signed-off-by: NPascal van Leeuwen <pvanleeuwen@verimatrix.com>
Acked-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When performing a transformation the hardware is given result
descriptors to save the result data. Those result descriptors are
batched using a 'first' and a 'last' bit. There are cases were more
descriptors than needed are given to the engine, leading to the engine
only using some of them, and not setting the last bit on the last
descriptor we gave. This causes issues were the driver and the hardware
aren't in sync anymore about the number of result descriptors given (as
the driver do not give a pool of descriptor to use for any
transformation, but a pool of descriptors to use *per* transformation).

This patch fixes it by attaching the number of given result descriptors
to the requests, and by using this number instead of the 'last' bit
found on the descriptors to process them.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Replace sg_nents_for_len by sg_nents when DMA mapping/unmapping buffers
and when looping over the SG entries. This fix cases where the SG
entries aren't used fully, which would in such cases led to using fewer
SG entries than needed (and thus the engine wouldn't have access to the
full input data and the result would be wrong).
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds support for retrieving intermediate IV from the crypto
engine when using the CBC block mode with AES and (3)DES. The retrieved
IV is copied to the request IV buffer, as requested by the kernel crypto
API.

This fix boot tests added by
commit 8efd972e ("crypto: testmgr - support checking skcipher output IV").
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Cosmetic patch removing an empty line in the skcipher token creation
routine.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch forbids the use of 2-key 3DES (K1 == K3) in FIPS mode.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

CRYPTO_TFM_REQ_WEAK_KEY confuses newcomers to the crypto API because it
sounds like it is requesting a weak key.  Actually, it is requesting
that weak keys be forbidden (for algorithms that have the notion of
"weak keys"; currently only DES and XTS do).

Also it is only one letter away from CRYPTO_TFM_RES_WEAK_KEY, with which
it can be easily confused.  (This in fact happened in the UX500 driver,
though just in some debugging messages.)

Therefore, make the intent clear by renaming it to
CRYPTO_TFM_REQ_FORBID_WEAK_KEYS.
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Remove the unnecessary setting of CRYPTO_ALG_TYPE_SKCIPHER.
Commit 2c95e6d9 ("crypto: skcipher - remove useless setting of type
flags") took care of this everywhere else, but a few more instances made
it into the tree at about the same time.  Squash them before they get
copy+pasted around again.

This patch shouldn't change any actual behavior.
Signed-off-by: NEric Biggers <ebiggers@google.com>
Acked-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In the cipher safexcel_send_req function, GCC warns that
first_rdesc may be used uninitialized. While this should never
happen, this patch removes the warning by initializing this
variable to NULL to make GCC happy.

This was reported by the kbuild test robot.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use the appropriate SPDX license identifiers and drop the license text.
This patch is only cosmetic.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Some skcipher algorithms set .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER.  But
this is redundant with the C structure type ('struct skcipher_alg'), and
crypto_register_skcipher() already sets the type flag automatically,
clearing any type flag that was already there.  Apparently the useless
assignment has just been copy+pasted around.

So, remove the useless assignment from all the skcipher algorithms.

This patch shouldn't change any actual behavior.
Signed-off-by: NEric Biggers <ebiggers@google.com>
Acked-by: NGilad Ben-Yossef <gilad@benyossef.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Some aead algorithms set .cra_flags = CRYPTO_ALG_TYPE_AEAD.  But this is
redundant with the C structure type ('struct aead_alg'), and
crypto_register_aead() already sets the type flag automatically,
clearing any type flag that was already there.  Apparently the useless
assignment has just been copy+pasted around.

So, remove the useless assignment from all the aead algorithms.

This patch shouldn't change any actual behavior.
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch main goal is to improve driver performance by moving the
crypto request from a list to a RDR ring shadow.

This is possible since there is one producer and one consume for this
RDR request shadow and one ring descriptor is left unused.
Doing this change eliminates the use of spinlock when accessing the
descriptor ring and the need to dynamicaly allocate memory per crypto
request.

The crypto request is placed in the first RDR shadow descriptor only
if there are enough descriptors, when the result handler is invoked,
it fetches the first result descriptor from RDR shadow.
Signed-off-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds support for two new algorithms in the Inside Secure
SafeXcel cryptographic engine driver: ecb(des3_ede) and cbc(des3_ede).
Signed-off-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds support for two algorithms in the Inside Secure SafeXcel
cryptographic engine driver: ecb(des) and cbc(des).
Signed-off-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds support for the eip197d engine to the Inside Secure
SafeXcel cryptographic driver. This new engine is similar to the eip197b
and reuse most of its code.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add a flags field in the private structure, and a first flag for engines
needing context invalidation (currently only the eip197b). The
invalidation is needed when the engine includes a TRC cache, which will
also be true for the upcoming addition of the eip197d engine.
Suggested-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

EIP engines do not support the same set of algorithms. So far the
supported engines in the Inside Secure SafeXcel driver support the same
set of algorithms, but that won't be true for all engines. This patch
adds an 'engines' field in the algorithm definitions so that they only
are registered when using a compatible cryptographic engine.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

At first we used two compatibles in the SafeXcel driver, named after the
engine revision: eip97 and eip197. However this family of engines has
more precise versions and in fact we're supporting the eip97ies and
eip197b. More versions will be supported in the future, such as the
eip197d, and we'll need to differentiate them.

This patch fixes the compatibles used in the driver, to now use precise
ones. The two historical compatibles are kept for backward
compatibility.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the authenc(hmac(sha384),cbc(aes)) algorithm support to
the Inside Secure SafeXcel driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the authenc(hmac(sha512),cbc(aes)) algorithm support to
the Inside Secure SafeXcel driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch reworks the way invalidation request handlers handle the
result descriptor errors, to use the common error handling function.
This improves the drivers in terms of readability and maintainability.
Suggested-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the authenc(hmac(sha1),cbc(aes)) AEAD algorithm
support to the Inside Secure SafeXcel driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the authenc(hmac(sha224),cbc(aes)) AEAD algorithm
support to the Inside Secure SafeXcel driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds support for the first AEAD algorithm in the Inside
Secure SafeXcel driver, authenc(hmac(sha256),cbc(aes)). As this is the
first AEAD algorithm added to this driver, common AEAD functions are
added as well.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch improves the error reporting from the Inside Secure driver to
the upper layers and crypto consumers. All errors reported by the engine
aren't fatal, and some may be genuine.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patches makes the key and context size computation dynamic when
using memzero_explicit() on these two arrays. This is safer, cleaner and
will help future modifications of the driver when these two parameters
sizes will changes (the context size will be bigger when using AEAD
algorithms).
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch makes the context control size computation dynamic, not to
rely on hardcoded values. This is better for the future, and will help
adding the AEAD support.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patches reworks the way the algorithm type is set in the context,
by using the fact that the decryption algorithms are just a combination
of the algorithm encryption type and CONTEXT_CONTROL_TYPE_NULL_IN.

This will help having simpler code when adding the AEAD support, to
avoid ending up with an endless switch case block.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch reworks the Inside Secure cipher functions, to remove all
skcipher specific information and structure from all functions generic
enough to be shared between skcipher and aead algorithms.

This is a cosmetic only patch.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch removes the use of VLAs to allocate requests on the stack, by
removing both SKCIPHER_REQUEST_ON_STACK and AHASH_REQUEST_ON_STACK. As
we still need to allocate requests on the stack to ease the creation of
invalidation requests a new, non-VLA, definition is used:
EIP197_REQUEST_ON_STACK.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The token used for encryption and decryption of skcipher algorithms sets
its stat field to "last packet". As it's a cipher only algorithm, there
is not hash operation and thus the "last hash" bit should be set to tell
the internal engine no hash operation should be performed.

This does not fix a bug, but improves the token definition to follow
exactly what's advised by the datasheet.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When exiting a transformation, the cra_exit() helper is called in each
driver providing one. The Inside Secure SafeXcel driver has one, which
is responsible of freeing some areas and of sending one invalidation
request to the crypto engine, to invalidate the context that was used
during the transformation.

We could see in some setups (when lots of transformations were being
used with a short lifetime, and hence lots of cra_exit() calls) NULL
pointer dereferences and other weird issues. All these issues were
coming from accessing the tfm context.

The issue is the invalidation request completion is checked using a
wait_for_completion_interruptible() call in both the cipher and hash
cra_exit() helpers. In some cases this was interrupted while the
invalidation request wasn't processed yet. And then cra_exit() returned,
and its caller was freeing the tfm instance. Only then the request was
being handled by the SafeXcel driver, which lead to the said issues.

This patch fixes this by using wait_for_completion() calls in these
specific cases.

Fixes: 1b44c5a6 ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The Inside Secure SafeXcel driver was firstly designed to support the
EIP197 cryptographic engine which is an evolution (with much more
feature, better performances) of the EIP97 cryptographic engine. This
patch convert the Inside Secure SafeXcel driver to support both engines
(EIP97 + EIP197).

The main differences are the register offsets and the context
invalidation process which is EIP197 specific. This patch adds an
indirection on the register offsets and adds checks not to send any
invalidation request when driving the EIP97. A new compatible is added
as well to bind the driver from device trees.
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch moves the request dequeueing into a workqueue to improve the
coalescing of interrupts when sending requests to the engine; as the
engine is capable of having one single interrupt for n requests sent.
Using a workqueue allows to send more request at once.
Suggested-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The SafeXcel context isn't used in the cache invalidation function. This
cosmetic patch removes it (as well as from the function prototype in the
header file and when the function is called).
Signed-off-by: NOfer Heifetz <oferh@marvell.com>
[Antoine: commit message]
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The cipher direction can be different for requests within the same
transformation context. This patch moves the direction flag from the
context to the request scope.
Signed-off-by: NOfer Heifetz <oferh@marvell.com>
[Antoine: commit message]
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>