This patch updates the TRC configuration so that the version of the
EIP197 engine being used is taken into account, as the configuration
differs between the EIP197B and the EIP197D.
Signed-off-by: NOfer Heifetz <oferh@marvell.com>
[Antoine: commit message]
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds support for the eip197d engine to the Inside Secure
SafeXcel cryptographic driver. This new engine is similar to the eip197b
and reuse most of its code.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

So far a single processing engine (PE) was configured and used in the
Inside Secure SafeXcel cryptographic engine driver. Some versions have
more than a single PE. This patch rework the driver's initialization to
take this into account and to allow configuring more than one PE.
Signed-off-by: NOfer Heifetz <oferh@marvell.com>
[Antoine: some reworks and commit message.]
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The Inside Secure SafeXcel driver currently uses 4 rings, but the
eip197d engines has 8 of them. This patch updates the driver so that
rings are allocated dynamically based on the number of available rings
supported by a given engine.
Signed-off-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add a flags field in the private structure, and a first flag for engines
needing context invalidation (currently only the eip197b). The
invalidation is needed when the engine includes a TRC cache, which will
also be true for the upcoming addition of the eip197d engine.
Suggested-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

EIP engines do not support the same set of algorithms. So far the
supported engines in the Inside Secure SafeXcel driver support the same
set of algorithms, but that won't be true for all engines. This patch
adds an 'engines' field in the algorithm definitions so that they only
are registered when using a compatible cryptographic engine.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

At first we used two compatibles in the SafeXcel driver, named after the
engine revision: eip97 and eip197. However this family of engines has
more precise versions and in fact we're supporting the eip97ies and
eip197b. More versions will be supported in the future, such as the
eip197d, and we'll need to differentiate them.

This patch fixes the compatibles used in the driver, to now use precise
ones. The two historical compatibles are kept for backward
compatibility.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the authenc(hmac(sha384),cbc(aes)) algorithm support to
the Inside Secure SafeXcel driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the hmac(sha384) algorithm support to the Inside Secure
SafeXcel driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the sha384 algorithm support to the Inside Secure
SafeXcel driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the authenc(hmac(sha512),cbc(aes)) algorithm support to
the Inside Secure SafeXcel driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the hmac(sha512) algorithm support to the Inside Secure
SafeXcel driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the sha512 algorithm support to the Inside Secure
SafeXcel driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

A counter is given to the engine when finishing hash computation. It
currently uses the blocksize while it counts the number of 64 bytes
blocks given to the engine. This works well for all algorithms so far,
as SHA1, SHA224 and SHA256 all have a blocksize of 64 bytes, but others
algorithms such as SHA512 wouldn't work.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the authenc(hmac(sha1),cbc(aes)) AEAD algorithm
support to the Inside Secure SafeXcel driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the authenc(hmac(sha224),cbc(aes)) AEAD algorithm
support to the Inside Secure SafeXcel driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds support for the first AEAD algorithm in the Inside
Secure SafeXcel driver, authenc(hmac(sha256),cbc(aes)). As this is the
first AEAD algorithm added to this driver, common AEAD functions are
added as well.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch improves the error reporting from the Inside Secure driver to
the upper layers and crypto consumers. All errors reported by the engine
aren't fatal, and some may be genuine.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This commit fixes the CONTEXT_CONTROL_TYPE_HASH_ENCRYPT_OUT and
CONTEXT_CONTROL_TYPE_HASH_DECRYPT_OUT types by assigning the right
value, and by renaming CONTEXT_CONTROL_TYPE_HASH_DECRYPT_OUT to
CONTEXT_CONTROL_TYPE_HASH_DECRYPT_IN.

This is not submitted as a fix for older kernel versions as these two
defines weren't used back then.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch removes the use of VLAs to allocate requests on the stack, by
removing both SKCIPHER_REQUEST_ON_STACK and AHASH_REQUEST_ON_STACK. As
we still need to allocate requests on the stack to ease the creation of
invalidation requests a new, non-VLA, definition is used:
EIP197_REQUEST_ON_STACK.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the hmac(sha224) support to the Inside Secure
cryptographic engine driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the hmac(sha256) support to the Inside Secure
cryptographic engine driver.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch fixes a typo in the EIP197_HIA_xDR_WR_CTRL_BUG register name,
as it should be EIP197_HIA_xDR_WR_CTRL_BUF. This is a cosmetic only
change.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Small cosmetic patch fixing one typo in the
EIP197_HIA_DSE_CFG_ALLWAYS_BUFFERABLE macro, it should be _ALWAYS_.
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

On Armada 7K/8K we need to explicitly enable the register clock. This
clock is optional because not all the SoCs using this IP need it but at
least for Armada 7K/8K it is actually mandatory.

The binding documentation is updated accordingly.
Signed-off-by: NGregory CLEMENT <gregory.clement@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In heavy traffic the DMA mapping is overwritten by multiple requests as
the DMA address is stored in a global context. This patch moves this
information to the per-hash request context so that it can't be
overwritten.

Fixes: 1b44c5a6 ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In heavy traffic the DMA mapping is overwritten by multiple requests as
the DMA address is stored in a global context. This patch moves this
information to the per-hash request context so that it can't be
overwritten.

Fixes: 1b44c5a6 ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: NOfer Heifetz <oferh@marvell.com>
[Antoine: rebased the patch, small fixes, commit message.]
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch updates the Inside Secure SafeXcel driver to avoid being
out-of-sync between the number of requests sent and the one being
completed.

The number of requests acknowledged by the driver can be different than
the threshold that was configured if new requests were being pushed to
the h/w in the meantime. The driver wasn't taking those into account,
and the number of remaining requests to handled (to reconfigure the
interrupt threshold) could be out-of sync.

This patch fixes it by not taking in account the number of requests
left, but by taking in account the total number of requests being sent
to the hardware, so that new requests are being taken into account.

Fixes: dc7e28a3 ("crypto: inside-secure - dequeue all requests at once")
Suggested-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The Inside Secure SafeXcel driver was firstly designed to support the
EIP197 cryptographic engine which is an evolution (with much more
feature, better performances) of the EIP97 cryptographic engine. This
patch convert the Inside Secure SafeXcel driver to support both engines
(EIP97 + EIP197).

The main differences are the register offsets and the context
invalidation process which is EIP197 specific. This patch adds an
indirection on the register offsets and adds checks not to send any
invalidation request when driving the EIP97. A new compatible is added
as well to bind the driver from device trees.
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The dequeueing function was putting back a request in the crypto queue
on failure (when not enough resources are available) which is not
perfect as the request will be handled much later. This patch updates
this logic by keeping a reference on the failed request to try
proceeding it later when enough resources are available.
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch modifies the result handling logic to continue handling
results when the completed requests counter is full and not showing the
actual number of requests to handle.
Suggested-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Increase the ring size to handle more requests in parallel, while
keeping the batch size (for interrupt coalescing) to its previous value.
The ring size and batch size are now unlinked.
Suggested-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch updates the dequeueing logic to dequeue all requests at once.
Since we can have many requests in the queue, the interrupt coalescing
is kept so that the ring interrupt fires every EIP197_MAX_BATCH_SZ at
most.

To allow dequeueing all requests at once while still using reasonable
settings for the interrupt coalescing, the result handling function was
updated to setup the threshold interrupt when needed (i.e. when more
requests than EIP197_MAX_BATCH_SZ are in the queue). When using this
capability the ring is marked as busy so that the dequeue function
enqueue new requests without setting the threshold interrupt.
Suggested-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch moves the result handling from an IRQ handler to a threaded
IRQ handler, to improve the number of complete requests being handled at
once.
Suggested-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch moves the request dequeueing into a workqueue to improve the
coalescing of interrupts when sending requests to the engine; as the
engine is capable of having one single interrupt for n requests sent.
Using a workqueue allows to send more request at once.
Suggested-by: NOfer Heifetz <oferh@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The SafeXcel context isn't used in the cache invalidation function. This
cosmetic patch removes it (as well as from the function prototype in the
header file and when the function is called).
Signed-off-by: NOfer Heifetz <oferh@marvell.com>
[Antoine: commit message]
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Increase the batch size to the maximum number of requests a ring can
handle at a time (its size). This is possible now that the request
queues are per hw ring. This improves performances.
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Update the inside-secure safexcel driver from using one global queue to
one queue per hw ring. This ease the request management and keep the hw
in sync with what's done in sw.
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Configure the data write bufferability to always buffer packets in the
DSE. This change slightly improves performance.
Signed-off-by: NIgal Liberman <igall@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When enable_single_wr is not enabled, the DSE will only write those
parts of a result descriptor that need updating, which means a final
result descriptor will be written in 2 or 3 smaller transfers.
When enable_single_wr is enabled the DSE will combine these 2-3
updates into one large write transfer, generally improving performance.
Signed-off-by: NIgal Liberman <igall@marvell.com>
Signed-off-by: NAntoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>