The AES GCM function (in ccp-ops) requires a fair amount of
stack space, which elicits a complaint when KASAN is enabled.
Rearranging and packing a few structures eliminates the
warning.
Signed-off-by: NGary R Hook <gary.hook@amd.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Endianness is dealt with when the command descriptor is
copied into the command queue. Remove any occurrences of
cpu_to_le32() found elsewhere.
Signed-off-by: NGary R Hook <gary.hook@amd.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add STM32 crypto support in stm32_defconfig file.
Signed-off-by: NFabien Dessenne <fabien.dessenne@st.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Enable the CRC (CRC32 crypto) on stm32746g-eval board
Signed-off-by: NFabien Dessenne <fabien.dessenne@st.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add CRC (CRC32 crypto) support to stm32f746.
Signed-off-by: NFabien Dessenne <fabien.dessenne@st.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This module registers a CRC32 ("Ethernet") and a CRC32C (Castagnoli)
algorithm that make use of the STMicroelectronics STM32 crypto hardware.

Theses algorithms are compatible with the little-endian generic ones.
Both algorithms use ~0 as default seed (key).
With CRC32C the output is xored with ~0.

Using TCRYPT CRC32C speed test, this shows up to 900% speedup compared
to the crc32c-generic algorithm.
Signed-off-by: NFabien Dessenne <fabien.dessenne@st.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Document device tree bindings for the STM32 CRC (crypto CRC32)
Signed-off-by: NFabien Dessenne <fabien.dessenne@st.com>
Acked-by: NRob Herring <robh@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Merge the crypto tree to resolve conflict between caam changes.

RNG instantiation was previously fixed by
commit 62743a41 ("crypto: caam - fix RNG init descriptor ret. code checking")
while deinstantiation was not addressed.

Since the descriptors used are similar, in the sense that they both end
with a JUMP HALT command, checking for errors should be similar too,
i.e. status code 7000_0000h should be considered successful.

Cc: <stable@vger.kernel.org> # 3.13+
Fixes: 1005bccd ("crypto: caam - enable instantiation of all RNG4 state handles")
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In case caam_jr_alloc() fails, ctx->dev carries the error code,
thus accessing it with dev_err() is incorrect.

Cc: <stable@vger.kernel.org> # 4.8+
Fixes: 8c419778 ("crypto: caam - add support for RSA algorithm")
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The way Job Ring platform devices are created and released does not
allow for multiple create-release cycles.

JR0 Platform device creation error
JR0 Platform device creation error
caam 2100000.caam: no queues configured, terminating
caam: probe of 2100000.caam failed with error -12

The reason is that platform devices are created for each job ring:

        for_each_available_child_of_node(nprop, np)
                if (of_device_is_compatible(np, "fsl,sec-v4.0-job-ring") ||
                    of_device_is_compatible(np, "fsl,sec4.0-job-ring")) {
                        ctrlpriv->jrpdev[ring] =
                                of_platform_device_create(np, NULL, dev);

which sets OF_POPULATED on the device node, but then it cleans these up:

        /* Remove platform devices for JobRs */
        for (ring = 0; ring < ctrlpriv->total_jobrs; ring++) {
                if (ctrlpriv->jrpdev[ring])
                        of_device_unregister(ctrlpriv->jrpdev[ring]);
        }

which leaves OF_POPULATED set.

Use of_platform_populate / of_platform_depopulate instead.
This allows for a bit of driver clean-up, jrpdev is no longer needed.

Logic changes a bit too:
-exit in case of_platform_populate fails, since currently even QI backend
depends on JR; true, we no longer support the case when "some" of the JR
DT nodes are incorrect
-when cleaning up, caam_remove() would also depopulate RTIC in case
it would have been populated somewhere else - not the case for now

Cc: <stable@vger.kernel.org>
Fixes: 313ea293 ("crypto: caam - Add Platform driver for Job Ring")
Reported-by: NRussell King <rmk+kernel@armlinux.org.uk>
Suggested-by: NRob Herring <robh+dt@kernel.org>
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Acked-by: NRob Herring <robh@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use sg_virt() instead of open-coding it.
Signed-off-by: NGeliang Tang <geliangtang@gmail.com>
Acked-by: NDavid S. Miller <davem@davemloft.net>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

An SGL to be initialized only once even when its buffers are written
to several times.
Signed-off-by: NStephan Mueller <smueller@chronox.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

3DES is missing the fips_allowed flag for CTR mode.
Signed-off-by: NMarcelo Henrique Cerri <marcelo.cerri@canonical.com>
Acked-by: NStephan Mueller <smueller@chronox.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

kernelci.org reports a build-time regression on linux-next, with a harmless
warning in x86 allmodconfig:

drivers/crypto/cavium/zip/zip_main.c:489:18: warning: format '%ld' expects argument of type 'long int', but argument 7 has type 'long long int' [-Wformat=]
drivers/crypto/cavium/zip/zip_main.c:489:18: warning: format '%ld' expects argument of type 'long int', but argument 6 has type 'long long int' [-Wformat=]
drivers/crypto/cavium/zip/zip_main.c:489:18: warning: format '%ld' expects argument of type 'long int', but argument 5 has type 'long long int' [-Wformat=]

The return type for atomic64_read() unfortunately differs between
architectures, with some defining it as atomic_long_read() and others
returning a 64-bit type explicitly. Fixing this in general would be nice,
but also require changing other users of these functions, so the simpler
workaround is to add a cast here that avoids the warnings on the default
build.

Fixes: 09ae5d37 ("crypto: zip - Add Compression/Decompression statistics")
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

There is a typo here.  It should be "stats" instead of "state".  The
impact is that we clear 224 bytes instead of 80 and we zero out memory
that we shouldn't.

Fixes: 09ae5d37 ("crypto: zip - Add Compression/Decompression statistics")
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add kernel-doc to s5p_aes_dev structure.
Signed-off-by: NKrzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The driver uses type of device (variant) only during probe so there is
no need to store it for later.
Signed-off-by: NKrzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Driver is capable of handling only one request at a time and it stores
it in its state container struct s5p_aes_dev.  This stored request must be
protected between concurrent invocations (e.g. completing current
request and scheduling new one).  Combination of lock and "busy" field
is used for that purpose.

When "busy" field is true, the driver will not accept new request thus
it will not overwrite currently handled data.

However commit 28b62b14 ("crypto: s5p-sss - Fix spinlock recursion
on LRW(AES)") moved some of the write to "busy" field out of a lock
protected critical section.  This might lead to potential race between
completing current request and scheduling a new one.  Effectively the
request completion might try to operate on new crypto request.

Cc: <stable@vger.kernel.org> # v4.10.x
Fixes: 28b62b14 ("crypto: s5p-sss - Fix spinlock recursion on LRW(AES)")
Signed-off-by: NKrzysztof Kozlowski <krzk@kernel.org>
Reviewed-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add support to submit ablkcipher and authenc algorithms
via the QI backend:
-ablkcipher:
cbc({aes,des,des3_ede})
ctr(aes), rfc3686(ctr(aes))
xts(aes)
-authenc:
authenc(hmac(md5),cbc({aes,des,des3_ede}))
authenc(hmac(sha*),cbc({aes,des,des3_ede}))

caam/qi being a new driver, let's wait some time to settle down without
interfering with existing caam/jr driver.
Accordingly, for now all caam/qi algorithms (caamalg_qi module) are
marked to be of lower priority than caam/jr ones (caamalg module).
Signed-off-by: NVakul Garg <vakul.garg@nxp.com>
Signed-off-by: NAlex Porosanu <alexandru.porosanu@nxp.com>
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

CAAM engine supports two interfaces for crypto job submission:
-job ring interface - already existing caam/jr driver
-Queue Interface (QI) - caam/qi driver added in current patch

QI is present in CAAM engines found on DPAA platforms.
QI gets its I/O (frame descriptors) from QMan (Queue Manager) queues.

This patch adds a platform device for accessing CAAM's queue interface.
The requests are submitted to CAAM using one frame queue per
cryptographic context. Each crypto context has one shared descriptor.
This shared descriptor is attached to frame queue associated with
corresponding driver context using context_a.

The driver hides the mechanics of FQ creation, initialisation from its
applications. Each cryptographic context needs to be associated with
driver context which houses the FQ to be used to transport the job to
CAAM. The driver provides API for:
(a) Context creation
(b) Job submission
(c) Context deletion
(d) Congestion indication - whether path to/from CAAM is congested

The driver supports affining its context to a particular CPU.
This means that any responses from CAAM for the context in question
would arrive at the given CPU. This helps in implementing one CPU
per packet round trip in IPsec application.

The driver processes CAAM responses under NAPI contexts.
NAPI contexts are instantiated only on cores with affined portals since
only cores having their own portal can receive responses from DQRR.

The responses from CAAM for all cryptographic contexts ride on a fixed
set of FQs. We use one response FQ per portal owning core. The response
FQ is configured in each core's and thus portal's dedicated channel.
This gives the flexibility to direct CAAM's responses for a crypto
context on a given core.
Signed-off-by: NVakul Garg <vakul.garg@nxp.com>
Signed-off-by: NAlex Porosanu <alexandru.porosanu@nxp.com>
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

A few other things need to be added in soc/qman, such that
caam/qi won't open-code them.
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Export qman_query_fq_np() function and related structures.
This will be needed in the caam/qi driver, where "queue empty"
condition will be decided based on the frm_cnt.
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add and export the ID of the channel serviced by the
CAAM (Cryptographic Acceleration and Assurance Module) DCP.
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Since qman_volatile_dequeue() is already exported, move the related
structures into the public header too.
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The md5_transform function is no longer used any where in the tree,
except for the crypto api's actual implementation of md5, so we can drop
the function from lib and put it as a static function of the crypto
file, where it belongs. There should be no new users of md5_transform,
anyway, since there are more modern ways of doing what it once achieved.
Signed-off-by: NJason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

aes_ctrby8_avx-x86_64.S uses the C preprocessor for token pasting
of character sequences that are not valid preprocessor tokens.
While this is allowed when preprocessing assembler files it exposes
an incompatibilty between the clang and gcc preprocessors where
clang does not strip leading white space from macro parameters,
leading to the CONCAT(%xmm, i) macro expansion on line 96 resulting
in a token with a space character embedded in it.

While this could be resolved by deleting the offending space character,
the assembler is perfectly capable of doing the token pasting correctly
for itself so we can just get rid of the preprocessor macros.
Signed-off-by: NMichael Davidson <md@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

A version 5 device provides the primitive commands
required for AES GCM. This patch adds support for
en/decryption.
Signed-off-by: NGary R Hook <gary.hook@amd.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Wire up support for Triple DES in ECB mode.
Signed-off-by: NGary R Hook <gary.hook@amd.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Incorporate 384-bit and 512-bit hashing for a version 5 CCP
device
Signed-off-by: NGary R Hook <gary.hook@amd.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

vpmsum implementations often don't kick in for short test vectors.
This is a simple test module that does a configurable number of
random tests, each up to 64kB and each with random offsets.

Both CRC-T10DIF and CRC32C are tested.

Cc: Anton Blanchard <anton@samba.org>
Signed-off-by: NDaniel Axtens <dja@axtens.net>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

T10DIF is a CRC16 used heavily in NVMe.

It turns out we can accelerate it with a CRC32 library and a few
little tricks.

Provide the accelerator based the refactored CRC32 code.

Cc: Anton Blanchard <anton@samba.org>
Thanks-to: Hong Bo Peng <penghb@cn.ibm.com>
Signed-off-by: NDaniel Axtens <dja@axtens.net>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When CRC32c was included in the kernel, Anton ripped out
the #ifdefs around reflected polynomials, because CRC32c
is always reflected. However, not all CRCs use reflection
so we'd like to make it optional.

Restore the REFLECT parts from Anton's original CRC32
implementation (https://github.com/antonblanchard/crc32-vpmsum)

That implementation is available under GPLv2+, so we're OK
from a licensing point of view:
https://github.com/antonblanchard/crc32-vpmsum/blob/master/LICENSE.TXT

As CRC32c requires REFLECT, add that #define.

Cc: Anton Blanchard <anton@samba.org>
Signed-off-by: NDaniel Axtens <dja@axtens.net>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The core nuts and bolts of the crc32c vpmsum algorithm will
also work for a number of other CRC algorithms with different
polynomials. Factor out the function into a new asm file.

To handle multiple users of the function, a user simply
provides constants, defines the name of their CRC function,
and then #includes the core algorithm file.

Cc: Anton Blanchard <anton@samba.org>
Signed-off-by: NDaniel Axtens <dja@axtens.net>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Merging 4.11-rc3 to pick up md5 removal from /dev/random.

In the generic XTS and LRW algorithms, for input data > 128 bytes, a
temporary buffer is allocated to hold the values to be XOR'ed with the
data before and after encryption or decryption.  If the allocation
fails, the fixed-size buffer embedded in the request buffer is meant to
be used as a fallback --- resulting in more calls to the ECB algorithm,
but still producing the correct result.  However, we weren't correctly
limiting subreq->cryptlen in this case, resulting in pre_crypt()
overrunning the embedded buffer.  Fix this by setting subreq->cryptlen
correctly.

Fixes: f1c131b4 ("crypto: xts - Convert to skcipher")
Fixes: 700cb3f5 ("crypto: lrw - Convert to skcipher")
Cc: stable@vger.kernel.org # v4.10+
Reported-by: NDmitry Vyukov <dvyukov@google.com>
Signed-off-by: NEric Biggers <ebiggers@google.com>
Acked-by: NDavid S. Miller <davem@davemloft.net>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The CCP registers its queues as channels capable of handling
general DMA operations. The NTB driver will use DMA if
directed, but as public channels can be reserved for use in
asynchronous operations some channels should be held back
as private. Since the public/private determination is
handled at a device level, reserve the "other" (secondary)
CCP channels as private.

Add a module parameter that allows for override, to be
applied to all channels on all devices.

CC: <stable@vger.kernel.org> # 4.10.x-
Signed-off-by: NGary R Hook <gary.hook@amd.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Under extremely heavy uses of padata, crashes occur, and with list
debugging turned on, this happens instead:

[87487.298728] WARNING: CPU: 1 PID: 882 at lib/list_debug.c:33
__list_add+0xae/0x130
[87487.301868] list_add corruption. prev->next should be next
(ffffb17abfc043d0), but was ffff8dba70872c80. (prev=ffff8dba70872b00).
[87487.339011]  [<ffffffff9a53d075>] dump_stack+0x68/0xa3
[87487.342198]  [<ffffffff99e119a1>] ? console_unlock+0x281/0x6d0
[87487.345364]  [<ffffffff99d6b91f>] __warn+0xff/0x140
[87487.348513]  [<ffffffff99d6b9aa>] warn_slowpath_fmt+0x4a/0x50
[87487.351659]  [<ffffffff9a58b5de>] __list_add+0xae/0x130
[87487.354772]  [<ffffffff9add5094>] ? _raw_spin_lock+0x64/0x70
[87487.357915]  [<ffffffff99eefd66>] padata_reorder+0x1e6/0x420
[87487.361084]  [<ffffffff99ef0055>] padata_do_serial+0xa5/0x120

padata_reorder calls list_add_tail with the list to which its adding
locked, which seems correct:

spin_lock(&squeue->serial.lock);
list_add_tail(&padata->list, &squeue->serial.list);
spin_unlock(&squeue->serial.lock);

This therefore leaves only place where such inconsistency could occur:
if padata->list is added at the same time on two different threads.
This pdata pointer comes from the function call to
padata_get_next(pd), which has in it the following block:

next_queue = per_cpu_ptr(pd->pqueue, cpu);
padata = NULL;
reorder = &next_queue->reorder;
if (!list_empty(&reorder->list)) {
       padata = list_entry(reorder->list.next,
                           struct padata_priv, list);
       spin_lock(&reorder->lock);
       list_del_init(&padata->list);
       atomic_dec(&pd->reorder_objects);
       spin_unlock(&reorder->lock);

       pd->processed++;

       goto out;
}
out:
return padata;

I strongly suspect that the problem here is that two threads can race
on reorder list. Even though the deletion is locked, call to
list_entry is not locked, which means it's feasible that two threads
pick up the same padata object and subsequently call list_add_tail on
them at the same time. The fix is thus be hoist that lock outside of
that block.
Signed-off-by: NJason A. Donenfeld <Jason@zx2c4.com>
Acked-by: NSteffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit 52060836 ("dt-bindings: omap-rng: Document SafeXcel IP-76
device variant") update the omap_rng Device Tree binding to add support
for the IP-76 variation of the IP. As part of this change, a "clocks"
property was added, but is indicated as "Required", without indicated
it's actually only required for some compatible strings.

This commit fixes that, by explicitly stating that the clocks property
is only required with the inside-secure,safexcel-eip76 compatible
string.

Fixes: 52060836 ("dt-bindings: omap-rng: Document SafeXcel IP-76 device variant")
Cc: <stable@vger.kernel.org>
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>