The of_find_device_by_node() takes a reference to the underlying device
structure, we should release that reference.

Fixes: 35af6403 ("crypto: caam - Check for CAAM block presence before registering with crypto layer")
Fixes: b189817c ("crypto: caam/qi - add ablkcipher and authenc algorithms")
Reviewed-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NWen Yang <yellowriver2010@hotmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit 0918f18c ("crypto: s5p - add AES support for Exynos5433")
introduced bug in dereferencing clk_names[1] on platforms different from
Exynos5433. On Exynos board XU3 call trace is:

"Unable to handle kernel paging request at virtual address 00004000"
(strcmp) from [<c0774014>] (of_property_match_string+0x58/0xd0)
...
(devm_clk_get) from [<c075c248>] (s5p_aes_probe+0xd4/0x4a0)
(s5p_aes_probe) from [<c059dbc4>] (platform_drv_probe+0x6c/0xa4)

Fix this by setting array clk_names size to 2.

Fixes: 0918f18c ("crypto: s5p - add AES support for Exynos5433")
Reported-by: NKrzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: NKamil Konieczny <k.konieczny@partner.samsung.com>
Reviewed-by: NKrzysztof Kozlowski <krzk@kernel.org>
Tested-by: NKrzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Some of these macros were conflicting with global namespace,
hence prefixing them with CXGB4.
Signed-off-by: NArjun Vynipadath <arjun@chelsio.com>
Signed-off-by: NVishal Kulkarni <vishal@chelsio.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add AES crypto HW acceleration for Exynos5433, with the help of SlimSSS IP.
Reviewed-by: NKrzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: NKamil Konieczny <k.konieczny@partner.samsung.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add an of_node_put when a tested device node is not available.

The semantic patch that fixes this problem is as follows
(http://coccinelle.lip6.fr):

// <smpl>
@@
identifier f;
local idexpression e;
expression x;
@@

e = f(...);
... when != of_node_put(e)
    when != x = e
    when != e = x
    when any
if (<+...of_device_is_available(e)...+>) {
  ... when != of_node_put(e)
(
  return e;
|
+ of_node_put(e);
  return ...;
)
}
// </smpl>

Fixes: 5343e674 ("crypto4xx: integrate ppc4xx-rng into crypto4xx")
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The cavium/zip implementation of the deflate compression algorithm is
incorrectly being registered under the generic driver name, which
prevents the generic implementation from being registered with the
crypto API when CONFIG_CRYPTO_DEV_CAVIUM_ZIP=y.  Similarly the lzs
algorithm (which does not currently have a generic implementation...)
is incorrectly being registered as lzs-generic.

Fix the naming collision by adding a suffix "-cavium" to the
cra_driver_name of the cavium/zip algorithms.

Fixes: 640035a2 ("crypto: zip - Add ThunderX ZIP driver core")
Cc: Mahipal Challa <mahipalreddy2006@gmail.com>
Cc: Jan Glauber <jglauber@cavium.com>
Signed-off-by: NEric Biggers <ebiggers@google.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

unlikely has already included in IS_ERR(), so just
remove redundant likely/unlikely annotation.
Signed-off-by: NChengguang Xu <cgxu519@gmx.com>
Reviewed-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix bug "s5p-sss crypto driver doesn't set next AES-CBC IV". While at this,
fix also AES-CTR mode. Tested on Odroid U3 with Eric Biggers branch
"iv-out-testing".
Signed-off-by: NKamil Konieczny <k.konieczny@partner.samsung.com>
Reported-by: NEric Biggers <ebiggers@kernel.org>
Reviewed-by: NKrzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When generating a split key or hashing the key, DMA mapping the key
buffers coming directly from user is incorrect, since they are
not guaranteed to be DMAable.

Update driver to first copy user-provided key in the output buffer
("key_out") and then use this buffer for in-place computation
(split key generation, respectively key hashing).
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix a side effect of adding xcbc support, which leads to DMA mapping the
key twice.

Fixes: 12b8567f ("crypto: caam - add support for xcbc(aes)")
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When driver started using state->caam_ctxt for storing both running hash
and final hash, it was not updated to handle different DMA unmap
lengths.

Cc: <stable@vger.kernel.org> # v4.19+
Fixes: c19650d6 ("crypto: caam - fix DMA mapping of stack memory")
Signed-off-by: NFranck LENORMAND <franck.lenormand@nxp.com>
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Replace hard coded AES block size with define.
Signed-off-by: NKrzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

There is no need to have the struct stm32_hash_dev *hdev static
since new value always be assigned before use it.
Signed-off-by: NMao Wenan <maowenan@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fixed Traffic Stall caused by
- Subcommands except last should have more bit set
- For esn case subcommand is required for linear skb only
- Also Optimized is_eth_imm usage
Signed-off-by: NAtul Gupta <atul.gupta@chelsio.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fixes gcc '-Wunused-but-set-variable' warning:

drivers/crypto/marvell/cipher.c: In function 'mv_cesa_skcipher_dma_req_init':
drivers/crypto/marvell/cipher.c:325:15: warning:
 variable 'ivsize' set but not used [-Wunused-but-set-variable]

It's not used any more after 0c99620f ("crypto: marvell - Use an unique
pool to copy results of requests")
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The current content of some of the driver messages and the way that they
are issued results in some confusion, especially in the area of the PSP
as it relates to SEV support. If SEV is not supported, a message is issued
that says "psp initialization failed." This makes it seem like there was
a problem, when in fact, the PSP support is just disabled if SEV is not
supported.

Update the driver to check SEV support a bit earlier and issue a debug-
level message if SEV is not supported, followed by a debug-level message
that the PSP is disabled. This way you will only see PSP messages if SEV
is supported or if debug information is desired. Also, remove the overall
"enabled" and "disabled" messages for the driver and rely on the CCP and
PSP support to issue component-specific messages.
Signed-off-by: NTom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

For chain mode in cipher(eg. AES-CBC/DES-CBC), the iv is continuously
updated in the operation. The new iv value should be written to device
register by software.
Reported-by: NEric Biggers <ebiggers@google.com>
Fixes: 433cd2c6 ("crypto: rockchip - add crypto driver for rk3288")
Cc: <stable@vger.kernel.org> # v4.5+
Signed-off-by: NZhang Zhijie <zhangzj@rock-chips.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In some cases, the nents of src scatterlist is different from
dst scatterlist. So two variables are used to handle the nents
of src&dst scatterlist.
Reported-by: NEric Biggers <ebiggers@google.com>
Fixes: 433cd2c6 ("crypto: rockchip - add crypto driver for rk3288")
Cc: <stable@vger.kernel.org> # v4.5+
Signed-off-by: NZhang Zhijie <zhangzj@rock-chips.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add missing break statement in order to prevent the code from falling
through to case S_DIN_to_DES.

This bug was found thanks to the ongoing efforts to enable
-Wimplicit-fallthrough.

Fixes: 63ee04c8 ("crypto: ccree - add skcipher support")
Cc: stable@vger.kernel.org
Signed-off-by: NGustavo A. R. Silva <gustavo@embeddedor.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add cpu_relax() in the loop that tries to enqueue the FDs.
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Avoid console being flooded with prints in case HW is too busy to accept
new enqueue requests.
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Driver was relying on an older DPIO API, which provided a CPU-affine DPIO
in case it was called with preemption disabled.

Since this is no longer the case, save the CPU-affine DPIO in per-cpu
private structure during setup and further use it on the hot path.

Note that preemption is no longer disabled while trying to enqueue an
FD. Thus it might be possible to run the enqueue on a different CPU
(due to migration, when in process context), however this wouldn't be a
functionality issue.
Since we allow for all cores to enqueue, we take care of data
structures setup to handle the case when number of (Rx, Tx) queue pairs
is smaller than number of cores.
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Last user of cdev_list_lock was removed in commit

  6422ccc5 ("crypto/chelsio/chtls: listen fails with multiadapt")

Cc: Atul Gupta <atul.gupta@chelsio.com>
Cc: Harsh Jain <harsh@chelsio.com>
Signed-off-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit 1358c13a ("crypto: ccree - fix resume race condition on init")
was missing a "inline" qualifier for stub function used when CONFIG_PM
is not set causing a build warning.

Fixes: 1358c13a ("crypto: ccree - fix resume race condition on init")
Cc: stable@kernel.org # v4.20
Signed-off-by: NGilad Ben-Yossef <gilad@benyossef.com>
Acked-by: NGeert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Modify setkey callback for cbc des and 3des to check for weak keys.
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add ecb mode support for aes, des, 3des and arc4 ciphers.
ecb(*) reuses existing skcipher implementation, updating it with support
for no IV.
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Some arc4 cipher algorithm defines show up in two places:
crypto/arc4.c and drivers/crypto/bcm/cipher.h.
Let's export them in a common header and update their users.
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The mapped_{src,dst}_nents _returned_ from the dma_map_sg call (which
could be less than src/dst_nents) have to be used to generate the aead,
skcipher job descriptors.
Signed-off-by: NIuliana Prodan <iuliana.prodan@nxp.com>
Reviewed-by: NHoria Geantă <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch removes an unused label.
Reported-by: NStephen Rothwell <sfr@canb.auug.org.au>
Fixes: f0fcf9ad ("crypto: qat - no need to check return...")
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

when the source sg contains more than 1 fragment and
destination sg contains 1 fragment, the caam driver
mishandle the buffers to be sent to caam.

Fixes: f2147b88 ("crypto: caam - Convert GCM to new AEAD interface")
Cc: <stable@vger.kernel.org> # 4.2+
Signed-off-by: NPankaj Gupta <pankaj.gupta@nxp.com>
Signed-off-by: NArun Pathak <arun.pathak@nxp.com>
Reviewed-by: NHoria Geanta <horia.geanta@nxp.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

A kexec reboot may leave the firmware in INIT or WORKING state.
Currently, we issue PLATFORM_INIT command during the probe without
checking the current state. The PLATFORM_INIT command fails if the
FW is already in INIT state. Lets check the current state, if FW
is not in UNINIT state then transition it to UNINIT before
initializing or upgrading the FW.
Signed-off-by: NBrijesh Singh <brijesh.singh@amd.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Gary Hook <gary.hook@amd.com>
Reviewed-by: NTom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

We were enabling autosuspend, which is using data set by the
hash module, prior to the hash module being inited, casuing
a crash on resume as part of the startup sequence if the race
was lost.

This was never a real problem because the PM infra was using low
res timers so we were always winning the race, until commit 8234f673
("PM-runtime: Switch autosuspend over to using hrtimers") changed that :-)

Fix this by seperating the PM setup and enablement and doing the
latter only at the end of the init sequence.
Signed-off-by: NGilad Ben-Yossef <gilad@benyossef.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: stable@kernel.org # v4.20
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Roland reports the following issue and provides a root cause analysis:

"On a v4.19 i.MX6 system with IMA and CONFIG_DMA_API_DEBUG enabled, a
warning is generated when accessing files on a filesystem for which IMA
measurement is enabled:

    ------------[ cut here ]------------
    WARNING: CPU: 0 PID: 1 at kernel/dma/debug.c:1181 check_for_stack.part.9+0xd0/0x120
    caam_jr 2101000.jr0: DMA-API: device driver maps memory from stack [addr=b668049e]
    Modules linked in:
    CPU: 0 PID: 1 Comm: switch_root Not tainted 4.19.0-20181214-1 #2
    Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
    Backtrace:
    [<c010efb8>] (dump_backtrace) from [<c010f2d0>] (show_stack+0x20/0x24)
    [<c010f2b0>] (show_stack) from [<c08b04f4>] (dump_stack+0xa0/0xcc)
    [<c08b0454>] (dump_stack) from [<c012b610>] (__warn+0xf0/0x108)
    [<c012b520>] (__warn) from [<c012b680>] (warn_slowpath_fmt+0x58/0x74)
    [<c012b62c>] (warn_slowpath_fmt) from [<c0199acc>] (check_for_stack.part.9+0xd0/0x120)
    [<c01999fc>] (check_for_stack.part.9) from [<c019a040>] (debug_dma_map_page+0x144/0x174)
    [<c0199efc>] (debug_dma_map_page) from [<c065f7f4>] (ahash_final_ctx+0x5b4/0xcf0)
    [<c065f240>] (ahash_final_ctx) from [<c065b3c4>] (ahash_final+0x1c/0x20)
    [<c065b3a8>] (ahash_final) from [<c03fe278>] (crypto_ahash_op+0x38/0x80)
    [<c03fe240>] (crypto_ahash_op) from [<c03fe2e0>] (crypto_ahash_final+0x20/0x24)
    [<c03fe2c0>] (crypto_ahash_final) from [<c03f19a8>] (ima_calc_file_hash+0x29c/0xa40)
    [<c03f170c>] (ima_calc_file_hash) from [<c03f2b24>] (ima_collect_measurement+0x1dc/0x240)
    [<c03f2948>] (ima_collect_measurement) from [<c03f0a60>] (process_measurement+0x4c4/0x6b8)
    [<c03f059c>] (process_measurement) from [<c03f0cdc>] (ima_file_check+0x88/0xa4)
    [<c03f0c54>] (ima_file_check) from [<c02d8adc>] (path_openat+0x5d8/0x1364)
    [<c02d8504>] (path_openat) from [<c02dad24>] (do_filp_open+0x84/0xf0)
    [<c02daca0>] (do_filp_open) from [<c02cf50c>] (do_open_execat+0x84/0x1b0)
    [<c02cf488>] (do_open_execat) from [<c02d1058>] (__do_execve_file+0x43c/0x890)
    [<c02d0c1c>] (__do_execve_file) from [<c02d1770>] (sys_execve+0x44/0x4c)
    [<c02d172c>] (sys_execve) from [<c0101000>] (ret_fast_syscall+0x0/0x28)
    ---[ end trace 3455789a10e3aefd ]---

The cause is that the struct ahash_request *req is created as a
stack-local variable up in the stack (presumably somewhere in the IMA
implementation), then passed down into the CAAM driver, which tries to
dma_single_map the req->result (indirectly via map_seq_out_ptr_result)
in order to make that buffer available for the CAAM to store the result
of the following hash operation.

The calling code doesn't know how req will be used by the CAAM driver,
and there could be other such occurrences where stack memory is passed
down to the CAAM driver. Therefore we should rather fix this issue in
the CAAM driver where the requirements are known."

Fix this problem by:
-instructing the crypto engine to write the final hash in state->caam_ctx
-subsequently memcpy-ing the final hash into req->result

Cc: <stable@vger.kernel.org> # v4.19+
Reported-by: NRoland Hieber <rhi@pengutronix.de>
Signed-off-by: NHoria Geantă <horia.geanta@nxp.com>
Tested-by: NRoland Hieber <rhi@pengutronix.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The header search path -I. in kernel Makefiles is very suspicious;
it allows the compiler to search for headers in the top of $(srctree),
where obviously no header file exists.

'git grep BCMDRIVER' has no hit. So, this macro is not referenced.

I was able to build this driver without the extra compiler options.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Currently, the Kbuild core manipulates header search paths in a crazy
way [1].

To fix this mess, I want all Makefiles to add explicit $(srctree)/ to
the search paths in the srctree. Some Makefiles are already written in
that way, but not all. The goal of this work is to make the notation
consistent, and finally get rid of the gross hacks.

Having whitespaces after -I does not matter since commit 48f6e3cf
("kbuild: do not drop -I without parameter").

[1]: https://patchwork.kernel.org/patch/9632347/Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The request unmap and bounce buffer copying is currently unnecessarily
done while holding the queue spin lock.
Signed-off-by: NLars Persson <larper@axis.com>
Signed-off-by: NVincent Whitchurch <rabinv@axis.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Avoid plain memcmp() on the AEAD tag value as this could leak
information through a timing side channel.
Signed-off-by: NLars Persson <larper@axis.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The implementation assumed that the client always wants the whole 16
byte AES-GCM tag. Now we respect the requested authentication tag size
fetched using crypto_aead_authsize().
Signed-off-by: NLars Persson <larper@axis.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The driver was optimized to only do cache maintenance for the last
word of the dma descriptor status array. Unfortunately an omission
also passed the last word as the address of the array start to the DMA
engine. In most cases this goes unnoticed since the hardware aligns
the address to a 64 byte boundary.
Signed-off-by: NLars Persson <larper@axis.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Clients may submit a new requests from the completion callback
context. The driver was not prepared to receive a request in this
state because it already held the request queue lock and a recursive
lock error is triggered.

Now all completions are queued up until we are ready to drop the queue
lock and then delivered.

The fault was triggered by TCP over an IPsec connection in the LTP
test suite:
  LTP: starting tcp4_ipsec02 (tcp_ipsec.sh -p ah -m transport -s "100 1000 65535")
  BUG: spinlock recursion on CPU#1, genload/943
   lock: 0xbf3c3094, .magic: dead4ead, .owner: genload/943, .owner_cpu: 1
  CPU: 1 PID: 943 Comm: genload Tainted: G           O    4.9.62-axis5-devel #6
  Hardware name: Axis ARTPEC-6 Platform
   (unwind_backtrace) from [<8010d134>] (show_stack+0x18/0x1c)
   (show_stack) from [<803a289c>] (dump_stack+0x84/0x98)
   (dump_stack) from [<8016e164>] (do_raw_spin_lock+0x124/0x128)
   (do_raw_spin_lock) from [<804de1a4>] (artpec6_crypto_submit+0x2c/0xa0)
   (artpec6_crypto_submit) from [<804def38>] (artpec6_crypto_prepare_submit_hash+0xd0/0x54c)
   (artpec6_crypto_prepare_submit_hash) from [<7f3165f0>] (ah_output+0x2a4/0x3dc [ah4])
   (ah_output [ah4]) from [<805df9bc>] (xfrm_output_resume+0x178/0x4a4)
   (xfrm_output_resume) from [<805d283c>] (xfrm4_output+0xac/0xbc)
   (xfrm4_output) from [<80587928>] (ip_queue_xmit+0x140/0x3b4)
   (ip_queue_xmit) from [<805a13b4>] (tcp_transmit_skb+0x4c4/0x95c)
   (tcp_transmit_skb) from [<8059f218>] (tcp_rcv_state_process+0xdf4/0xdfc)
   (tcp_rcv_state_process) from [<805a7530>] (tcp_v4_do_rcv+0x64/0x1ac)
   (tcp_v4_do_rcv) from [<805a9724>] (tcp_v4_rcv+0xa34/0xb74)
   (tcp_v4_rcv) from [<80581d34>] (ip_local_deliver_finish+0x78/0x2b0)
   (ip_local_deliver_finish) from [<8058259c>] (ip_local_deliver+0xe4/0x104)
   (ip_local_deliver) from [<805d23ec>] (xfrm4_transport_finish+0xf4/0x144)
   (xfrm4_transport_finish) from [<805df564>] (xfrm_input+0x4f4/0x74c)
   (xfrm_input) from [<804de420>] (artpec6_crypto_task+0x208/0x38c)
   (artpec6_crypto_task) from [<801271b0>] (tasklet_action+0x60/0xec)
   (tasklet_action) from [<801266d4>] (__do_softirq+0xcc/0x3a4)
   (__do_softirq) from [<80126d20>] (irq_exit+0xf4/0x15c)
   (irq_exit) from [<801741e8>] (__handle_domain_irq+0x68/0xbc)
   (__handle_domain_irq) from [<801014f0>] (gic_handle_irq+0x50/0x94)
   (gic_handle_irq) from [<80657370>] (__irq_usr+0x50/0x80)
Signed-off-by: NLars Persson <larper@axis.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>