This implements the ECB, CBC, CTR and XTS asynchronous block ciphers
using the AArch32 versions of the ARMv8 Crypto Extensions for AES.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This implements the SHA-224/256 secure hash algorithm using the AArch32
versions of the ARMv8 Crypto Extensions for SHA2.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This implements the SHA1 secure hash algorithm using the AArch32
versions of the ARMv8 Crypto Extensions for SHA1.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This moves all Kconfig symbols defined in crypto/Kconfig that depend
on CONFIG_ARM to a dedicated Kconfig file in arch/arm/crypto, which is
where the code that implements those features resides as well.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Even if bus is not hot-pluggable, the devices can be bound and unbound
from the driver via sysfs, so we should not be using __init/__exit
annotations on probe() and remove() methods. The only exception is
drivers registered with platform_driver_probe() which specifically
disables sysfs bind/unbind attributes.
Signed-off-by: NDmitry Torokhov <dmitry.torokhov@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

PCI bus is hot-pluggable, and even if it wasn't one can still unbind the
device from driver via sysfs, so we should not make driver's remove
method as __exit.
Signed-off-by: NDmitry Torokhov <dmitry.torokhov@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Even if bus is not hot-pluggable, the devices can be unbound from the
driver via sysfs, so we should not be using __exit annotations on
remove() methods. The only exception is drivers registered with
platform_driver_probe() which specifically disables sysfs bind/unbind
attributes.

Similarly probe() methods should not be marked __init unless
platform_driver_probe() is used.
Signed-off-by: NDmitry Torokhov <dmitry.torokhov@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Even if bus is not hot-pluggable, the devices can be unbound from the
driver via sysfs, so we should not be using __exit annotations on
remove() methods. The only exception is drivers registered with
platform_driver_probe() which specifically disables sysfs bind/unbind
attributes
Signed-off-by: NDmitry Torokhov <dmitry.torokhov@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Even if bus is not hot-pluggable, the devices can be unbound from the
driver via sysfs, so we should not be using __exit annotations on
remove() methods. The only exception is drivers registered with
platform_driver_probe() which specifically disables sysfs bind/unbind
attributes.
Signed-off-by: NDmitry Torokhov <dmitry.torokhov@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit 5be4d4c9 ("crypto: replace scatterwalk_sg_next with sg_next")
did not consider the fact that scatterwalk_sg_next() was looking at
sg entry length, while sg_next() looks at the "chained" sg bit.

This should have no effect in theory. However in practice, there are
cases where the sg table is initialized to a number of entries and
some of them are not properly configured. While scatterwalk_sg_next()
would have returned NULL (since sg length = 0 and sg page_link = 0),
sg_next() happily returns the next unconfigured sg entry.

insmod tcrypt.ko mode=500 sec=1

testing speed of async cbc(aes) (cbc-aes-talitos) encryption
test 0 (128 bit key, 16 byte blocks):
Unable to handle kernel paging request for data at address 0x00000000
Faulting instruction address: 0xc00d79e4
Oops: Kernel access of bad area, sig: 11 [#1]
SMP NR_CPUS=8 P1022 DS
Modules linked in: tcrypt(+) talitos
CPU: 0 PID: 2670 Comm: insmod Not tainted 4.0.0-rc1-QorIQ-SDK-V1.6+g904f1ca82209 #1
task: e8de3200 ti: e70bc000 task.ti: e70bc000
NIP: c00d79e4 LR: f92d223c CTR: c00d79c8
REGS: e70bda00 TRAP: 0300   Not tainted  (4.0.0-rc1-QorIQ-SDK-V1.6+g904f1ca82209)
MSR: 00029000 <CE,EE,ME>  CR: 84428f22  XER: 00000000
DEAR: 00000000 ESR: 00000000
GPR00: f92d223c e70bdab0 e8de3200 00000000 e70bdbb8 00000001 00000000 00000000
GPR08: 00000000 00000000 c08b0380 27282010 c00d79c8 1003a634 00000000 e70bdf1c
GPR16: e70bdef0 00000020 00000000 c08c0000 00000010 00000000 e70bdbb8 00000010
GPR24: e976d3a8 00000010 00000000 e70bdbd8 e8961010 00000001 c086e560 00000000
NIP [c00d79e4] page_address+0x1c/0x110
LR [f92d223c] talitos_map_sg+0x130/0x184 [talitos]
Call Trace:
[e70bdab0] [00000010] 0x10 (unreliable)
[e70bdad0] [f92d223c] talitos_map_sg+0x130/0x184 [talitos]
[e70bdb00] [f92d30d8] common_nonsnoop.constprop.13+0xc0/0x304 [talitos]
[e70bdb30] [f933fd90] test_acipher_speed+0x434/0x7dc [tcrypt]
[e70bdcc0] [f934318c] do_test+0x2478/0x306c [tcrypt]
[e70bdd80] [f11fe058] tcrypt_mod_init+0x58/0x100 [tcrypt]
[e70bdda0] [c0002354] do_one_initcall+0x90/0x1f4
[e70bde10] [c061fe00] do_init_module+0x60/0x1ac
[e70bde30] [c00a79f0] load_module+0x185c/0x1f88
[e70bdee0] [c00a82b0] SyS_finit_module+0x7c/0x98
[e70bdf40] [c000e8b0] ret_from_syscall+0x0/0x3c
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Enable user to select OCTEON SHA1/256/512 modules.
Signed-off-by: NAaro Koskinen <aaro.koskinen@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add OCTEON SHA512 module.
Signed-off-by: NAaro Koskinen <aaro.koskinen@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add OCTEON SHA256 module.
Signed-off-by: NAaro Koskinen <aaro.koskinen@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add OCTEON SHA1 module.
Signed-off-by: NAaro Koskinen <aaro.koskinen@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add instruction definitions for SHA1/256/512.
Signed-off-by: NAaro Koskinen <aaro.koskinen@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Always disable preemption on behalf of the drivers when crypto engine
is taken into use. This will simplify the usage.
Signed-off-by: NAaro Koskinen <aaro.koskinen@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Don't disable bottom half while the crypto engine is in use, as it
should be unnecessary: All kernel crypto engine usage is wrapped with
crypto engine state save/restore, so if we get interrupted by softirq
that uses crypto they should save and restore our context.

This actually fixes an issue when running OCTEON MD5 with interrupts
disabled (tcrypt mode=302). There's a WARNING because the module is
trying to enable the bottom half with irqs disabled:

[   52.656610] ------------[ cut here ]------------
[   52.661439] WARNING: CPU: 1 PID: 428 at /home/aaro/git/linux/kernel/softirq.c:150 __local_bh_enable_ip+0x9c/0xd8()
[   52.671780] Modules linked in: tcrypt(+)
[...]
[   52.763539] [<ffffffff8114082c>] warn_slowpath_common+0x94/0xd8
[   52.769465] [<ffffffff81144614>] __local_bh_enable_ip+0x9c/0xd8
[   52.775390] [<ffffffff81119574>] octeon_md5_final+0x12c/0x1e8
[   52.781144] [<ffffffff81337050>] shash_compat_digest+0xd0/0x1b0
Signed-off-by: NAaro Koskinen <aaro.koskinen@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The current cryptodev-2.6 tree commits:

d9850fc5 ("crypto: powerpc/sha1 - kernel config")
50ba29aa ("crypto: powerpc/sha1 - glue")

failed to properly place files under arch/powerpc/crypto, which
leads to build errors:

make[1]: *** No rule to make target 'arch/powerpc/crypto/sha1-spe-asm.o', needed by 'arch/powerpc/crypto/sha1-ppc-spe.o'.  Stop.
make[1]: *** No rule to make target 'arch/powerpc/crypto/sha1_spe_glue.o', needed by 'arch/powerpc/crypto/sha1-ppc-spe.o'.  Stop.
Makefile:947: recipe for target 'arch/powerpc/crypto' failed

Move the two sha1 spe files under crypto/, and whilst there, rename
other powerpc crypto files with underscores to use dashes for
consistency.

Cc: Markus Stockhausen <stockhausen@collogia.de>
Signed-off-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This adds ACPI support for APM X-Gene RNG unit.
Signed-off-by: NFeng Kan <fkan@apm.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The file Documentation/DocBook/crypto-API.tmpl documents the kernel
crypto API and is maintained.
Signed-off-by: NStephan Mueller <smueller@chronox.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The patch moves the information provided in
Documentation/crypto/crypto-API-userspace.txt into a separate chapter in
the kernel crypto API DocBook. Some corrections are applied (such as
removing a reference to Netlink when the AF_ALG socket is referred to).

In addition, the AEAD and RNG interface description is now added.

Also, a brief description of the zero-copy interface with an example
code snippet is provided.
Signed-off-by: NStephan Mueller <smueller@chronox.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Change the RNGs to always return 0 in success case.

This patch ensures that seqiv.c works with RNGs other than krng. seqiv
expects that any return code other than 0 is an error. Without the
patch, rfc4106(gcm(aes)) will not work when using a DRBG or an ANSI
X9.31 RNG.
Signed-off-by: NStephan Mueller <smueller@chronox.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix rng_unmap_ctx's DMA_UNMAP size problem for caam_rng, else system would
report the below calltrace during cleanup caam_rng.
Since rng_create_sh_desc() creates a fixed descriptor of exactly 4
command-lengths now, also update DESC_RNG_LEN to (4 * CAAM_CMD_SZ).

caam_jr ffe301000.jr: DMA-API: device driver frees DMA memory with different size [device address=0x000000007f080010] [map size=16 bytes] [unmap size=40 bytes]
------------[ cut here ]------------
WARNING: at lib/dma-debug.c:887
Modules linked in:
task: c0000000f7cdaa80 ti: c0000000e5340000 task.ti: c0000000e5340000
NIP: c0000000004f5bc8 LR: c0000000004f5bc4 CTR: c0000000005f69b0
REGS: c0000000e53433c0 TRAP: 0700   Not tainted
MSR: 0000000080029000 <CE,EE,ME>  CR: 24088482  XER: 00000000
SOFTE: 0

GPR00: c0000000004f5bc4 c0000000e5343640 c0000000012af360 000000000000009f
GPR04: 0000000000000000 00000000000000a0 c000000000d02070 c000000015980660
GPR08: c000000000cff360 0000000000000000 0000000000000000 c0000000012da018
GPR12: 00000000000001e3 c000000001fff780 00000000100f0000 0000000000000001
GPR16: 0000000000000002 0000000000000000 0000000000000000 0000000000000000
GPR20: 0000000000000000 0000000000000000 ffffffffffffffff 0000000000000001
GPR24: 0000000000000001 0000000000000001 0000000000000000 0000000000000001
GPR28: c000000001556b90 c000000001565b80 c0000000e5343750 c0000000f9427480
NIP [c0000000004f5bc8] .check_unmap+0x538/0x9c0
LR [c0000000004f5bc4] .check_unmap+0x534/0x9c0
Call Trace:
[c0000000e5343640] [c0000000004f5bc4] .check_unmap+0x534/0x9c0 (unreliable)
[c0000000e53436e0] [c0000000004f60d4] .debug_dma_unmap_page+0x84/0xb0
[c0000000e5343810] [c00000000082f9d4] .caam_cleanup+0x1d4/0x240
[c0000000e53438a0] [c00000000056cc88] .hwrng_unregister+0xd8/0x1c0
Instruction dump:
7c641b78 41de0410 e8a90050 2fa50000 419e0484 e8de0028 e8ff0030 3c62ff90
e91e0030 38638388 48546ed9 60000000 <0fe00000> 3c62ff8f 38637fc8 48546ec5
---[ end trace e43fd1734d6600df ]---
Signed-off-by: NYanjiang Jin <yanjiang.jin@windriver.com>
Acked-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

sec4_sg_bytes not being properly initialized causes ahash_done
to try to free unallocated DMA memory:

caam_jr ffe301000.jr: DMA-API: device driver tries to free DMA memory it has not allocated [device address=0xdeadbeefdeadbeef] [size=3735928559 bytes]
------------[ cut here ]------------
WARNING: at lib/dma-debug.c:1093
Modules linked in:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.0.0-rc1+ #6
task: e9598c00 ti: effca000 task.ti: e95a2000
NIP: c04ef24c LR: c04ef24c CTR: c0549730
REGS: effcbd40 TRAP: 0700   Not tainted  (4.0.0-rc1+)
MSR: 00029002 <CE,EE,ME>  CR: 22008084  XER: 20000000

GPR00: c04ef24c effcbdf0 e9598c00 00000096 c08f7424 c00ab2b0 00000000 00000001
GPR08: c0fe7510 effca000 00000000 000001c3 22008082 00000000 c1048e77 c1050000
GPR16: c0c36700 493c0040 0000002c e690e4a0 c1054fb4 c18bac40 00029002 c18b0788
GPR24: 00000014 e690e480 effcbe48 00000000 c0fde128 e6ffac10 deadbeef deadbeef
NIP [c04ef24c] check_unmap+0x93c/0xb40
LR [c04ef24c] check_unmap+0x93c/0xb40
Call Trace:
[effcbdf0] [c04ef24c] check_unmap+0x93c/0xb40 (unreliable)
[effcbe40] [c04ef4f4] debug_dma_unmap_page+0xa4/0xc0
[effcbec0] [c070cda8] ahash_done+0x128/0x1a0
[effcbef0] [c0700070] caam_jr_dequeue+0x1d0/0x290
[effcbf40] [c0045f40] tasklet_action+0x110/0x1f0
[effcbf80] [c0044bc8] __do_softirq+0x188/0x700
[effcbfe0] [c00455d8] irq_exit+0x108/0x120
[effcbff0] [c000f520] call_do_irq+0x24/0x3c
[e95a3e20] [c00059b8] do_IRQ+0xc8/0x170
[e95a3e50] [c0011bc8] ret_from_except+0x0/0x18
Signed-off-by: NYanjiang Jin <yanjiang.jin@windriver.com>
Acked-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This adds a driver for random number generator present on Broadcom
IPROC devices.
Reviewed-by: NRay Jui <rjui@broadcom.com>
Signed-off-by: NScott Branden <sbranden@broadcom.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Documents the IPROC random number generator device tree bindings
used in some Broadcom chipsets.
Reviewed-by: NRay Jui <rjui@broadcom.com>
Signed-off-by: NScott Branden <sbranden@broadcom.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit 7e933d3b ("crypto: ux500: use dmaengine_prep_slave_sg
API") changed the code to use the new API, but forgot to update an error
message.
Signed-off-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This is properly defined in the md5 header file.
Signed-off-by: NMartin Hicks <mort@bork.org>
Acked-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

There were multiple loops in a row, for each separate step of the
initialization of the channels.  Simplify to a single loop.
Signed-off-by: NMartin Hicks <mort@bork.org>
Acked-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The DRBG code contains memset(0) calls to initialize a varaible
that are not necessary as the variable is always overwritten by
the processing.

This patch increases the CTR and Hash DRBGs by about 5%.
Signed-off-by: NStephan Mueller <smueller@chronox.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The CTR DRBG only encrypts one single block at a time. Thus, use the
single block crypto API to avoid additional overhead from the block
chaining modes.

With the patch, the speed of the DRBG increases between 30% and 40%.

The DRBG still passes the CTR DRBG CAVS test.
Signed-off-by: NStephan Mueller <smueller@chronox.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Integrate the module into the kernel config tree.
Signed-off-by: NMarkus Stockhausen <stockhausen@collogia.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Glue code for crypto infrastructure. Call the assembler
code where required. Take a little care about small input
data. Kick out early for input chunks < 64 bytes and replace
memset for context cleanup with simple loop.
Signed-off-by: NMarkus Stockhausen <stockhausen@collogia.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This is the assembler code for the MD5 implementation.
Handling of algorithm constants has been slightly
changed to reduce register usage and make better use
of cores with multiple ALUs. Thus they are stored as
delta values.
Signed-off-by: NMarkus Stockhausen <stockhausen@collogia.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix typo, "intialization" -> "initialization"
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Enable compilation of the AEAD AF_ALG support and provide a Kconfig
option to compile the AEAD AF_ALG support.
Signed-off-by: NStephan Mueller <smueller@chronox.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the AEAD support for AF_ALG.

The implementation is based on algif_skcipher, but contains heavy
modifications to streamline the interface for AEAD uses.

To use AEAD, the user space consumer has to use the salg_type named
"aead".

The AEAD implementation includes some overhead to calculate the size of
the ciphertext, because the AEAD implementation of the kernel crypto API
makes implied assumption on the location of the authentication tag. When
performing an encryption, the tag will be added to the created
ciphertext (note, the tag is placed adjacent to the ciphertext). For
decryption, the caller must hand in the ciphertext with the tag appended
to the ciphertext. Therefore, the selection of the used memory
needs to add/subtract the tag size from the source/destination buffers
depending on the encryption type. The code is provided with comments
explaining when and how that operation is performed.

A fully working example using all aspects of AEAD is provided at
http://www.chronox.de/libkcapi.htmlSigned-off-by: NStephan Mueller <smueller@chronox.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The kernel crypto API has many indirections which warrant a description
as otherwise one can get easily lost. The description explains the
layers of the kernel crypto API based on examples.
Signed-off-by: NStephan Mueller <smueller@chronox.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Integrate the module into the kernel config tree.
Signed-off-by: NMarkus Stockhausen <stockhausen@collogia.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Glue code for crypto infrastructure. Call the assembler
code where required. Disable preemption during calculation
and enable SPE instructions in the kernel prior to the
call. Avoid to disable preemption for too long.

Take a little care about small input data. Kick out early
for input chunks < 64 bytes and replace memset for context
cleanup with simple loop.
Signed-off-by: NMarkus Stockhausen <stockhausen@collogia.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>