Merge crypto to resolve conflict in crypto/Kconfig.

This patch adds unaligned buffer tests for hashes.

The first new test is with one byte offset and the second test checks if
cra_alignmask for driver is big enough; for example, for testing a case
where cra_alignmask is set to 7, but driver really needs buffers to be
aligned to 16 bytes.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds unaligned buffer tests for AEADs.

The first new test is with one byte offset and the second test checks if
cra_alignmask for driver is big enough; for example, for testing a case
where cra_alignmask is set to 7, but driver really needs buffers to be
aligned to 16 bytes.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds unaligned buffer tests for blkciphers.

The first new test is with one byte offset and the second test checks if
cra_alignmask for driver is big enough; for example, for testing a case
where cra_alignmask is set to 7, but driver really needs buffers to be
aligned to 16 bytes.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Patch adds check for alg_test_descs list order, so that accidentically
misplaced entries are found quicker. Duplicate entries are also checked for.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This reverts commit cf1521a1.

Instruction (vpgatherdd) that this implementation relied on turned out to be
slow performer on real hardware (i5-4570). The previous 8-way twofish/AVX
implementation is therefore faster and this implementation should be removed.

Converting this implementation to use the same method as in twofish/AVX for
table look-ups would give additional ~3% speed up vs twofish/AVX, but would
hardly be worth of the added code and binary size.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This reverts commit 60488010.

Instruction (vpgatherdd) that this implementation relied on turned out to be
slow performer on real hardware (i5-4570). The previous 4-way blowfish
implementation is therefore faster and this implementation should be removed.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add implementation tuned for more performance on real hardware. Changes are
mostly around the part mixing 128-bit extract and insert instructions and
AES-NI instructions. Also 'vpbroadcastb' instructions have been change to
'vpshufb with zero mask'.

Tests on Intel Core i5-4570:

tcrypt ECB results, old-AVX2 vs new-AVX2:

size    128bit key      256bit key
        enc     dec     enc     dec
256     1.00x   1.00x   1.00x   1.00x
1k      1.08x   1.09x   1.05x   1.06x
8k      1.06x   1.06x   1.06x   1.06x

tcrypt ECB results, AVX vs new-AVX2:

size    128bit key      256bit key
        enc     dec     enc     dec
256     1.00x   1.00x   1.00x   1.00x
1k      1.51x   1.50x   1.52x   1.50x
8k      1.47x   1.48x   1.48x   1.48x
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The MODULE_LICENSE macro invocation must use either "GPL" or "GPL v2",
but not "GPLv2" in order to be detected by the module loader.

This fixes the allmodconfig build error:

FATAL: modpost: GPL-incompatible module bcm2835-rng.ko uses GPL-only symbol 'platform_driver_unregister'
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Cc: Dom Cobley <popcornmix@gmail.com>
Cc: Lubomir Rintel <lkundrak@v3.sk>
Cc: Stephen Warren <swarren@wwwdotorg.org>
Cc: Matt Mackall <mpm@selenic.com>
Cc: linux-rpi-kernel@lists.infradead.org
Acked-by: NLubomir Rintel <lkundrak@v3.sk>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The new XTS code for aesni_intel uses input buffers directly as memory operands
for pxor instructions, which causes crash if those buffers are not aligned to
16 bytes.

Patch changes XTS code to handle unaligned memory correctly, by loading memory
with movdqu instead.
Reported-by: NDave Jones <davej@redhat.com>
Tested-by: NDave Jones <davej@redhat.com>
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The Nomadik HW RNG driver has seen some rust and is not preparing
the clock before use. Fix this up so we get rid of runtime
complaints from the clock subsystem.
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The usage of strict_strtoul() is not preferred, because
strict_strtoul() is obsolete. Thus, kstrtoul() should be
used.
Signed-off-by: NJingoo Han <jg1.han@samsung.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

These symbols are referenced only in this file and hence
should be static.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Tested-by: NTobias Rauter <tobiasrauter@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use NULL instead of 0 for pointer variables.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Tested-by: NTobias Rauter <tobiasrauter@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

devm_* APIs are device managed and make cleanup and exit
code simpler.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Tested-by: NTobias Rauter <tobiasrauter@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit 0998d063 (device-core: Ensure drvdata = NULL when no
driver is bound) removes the need to set driver data field to
NULL.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Tested-by: NTobias Rauter <tobiasrauter@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use the wrapper functions for getting and setting the driver data using
platform_device instead of using dev_{get,set}_drvdata() with &pdev->dev,
so we can directly pass a struct platform_device.

Also, unnecessary dev_set_drvdata() is removed, because the driver core
clears the driver data to NULL after device_release or on probe failure.
Signed-off-by: NJingoo Han <jg1.han@samsung.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Calling runtime PM API for every block causes serious perf hit to
crypto operations that are done on a long buffer.
As crypto is performed on a page boundary, encrypting large buffers can
cause a series of crypto operations divided by page. The runtime PM API
is also called those many times.

We call runtime_pm_get_sync only at beginning on the session (cra_init)
and runtime_pm_put at the end. This result in upto a 50% speedup as below.
This doesn't make the driver to keep the system awake as runtime get/put
is only called during a crypto session which completes usually quickly.

Before:
root@beagleboard:~# time -v openssl speed -evp aes-128-cbc
Doing aes-128-cbc for 3s on 16 size blocks: 13310 aes-128-cbc's in 0.01s
Doing aes-128-cbc for 3s on 64 size blocks: 13040 aes-128-cbc's in 0.04s
Doing aes-128-cbc for 3s on 256 size blocks: 9134 aes-128-cbc's in 0.03s
Doing aes-128-cbc for 3s on 1024 size blocks: 8939 aes-128-cbc's in 0.01s
Doing aes-128-cbc for 3s on 8192 size blocks: 4299 aes-128-cbc's in 0.00s

After:
root@beagleboard:~# time -v openssl speed -evp aes-128-cbc
Doing aes-128-cbc for 3s on 16 size blocks: 18911 aes-128-cbc's in 0.02s
Doing aes-128-cbc for 3s on 64 size blocks: 18878 aes-128-cbc's in 0.02s
Doing aes-128-cbc for 3s on 256 size blocks: 11878 aes-128-cbc's in 0.10s
Doing aes-128-cbc for 3s on 1024 size blocks: 11538 aes-128-cbc's in 0.05s
Doing aes-128-cbc for 3s on 8192 size blocks: 4857 aes-128-cbc's in 0.03s

While at it, also drop enter and exit pr_debugs, in related code. tracers
can be used for that.

Tested on a Beaglebone (AM335x SoC) board.
Signed-off-by: NJoel A Fernandes <joelagnel@ti.com>
Acked-by: NKevin Hilman <khilman@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The sahara crypto driver has an incorrect MODULE_DEVICE_TABLE, which
prevents us from actually building this driver as a loadable module.

sahara_dt_ids is a of_device_id array, so we have to use
MODULE_DEVICE_TABLE(of, ...).
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Cc: Javier Martin <javier.martin@vista-silicon.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

It appears that the performance of 'vpgatherdd' is suboptimal for this kind of
workload (tested on Core i5-4570) and causes blowfish-avx2 to be significantly
slower than blowfish-amd64. So disable the AVX2 implementation to avoid
performance regressions.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

It appears that the performance of 'vpgatherdd' is suboptimal for this kind of
workload (tested on Core i5-4570) and causes twofish_avx2 to be significantly
slower than twofish_avx. So disable the AVX2 implementation to avoid
performance regressions.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

lib/crc-t10dif.c:42:1-3: WARNING: PTR_RET can be used

 Use PTR_RET rather than if(IS_ERR(...)) + PTR_ERR

Generated by: coccinelle/api/ptr_ret.cocci
Reported-by: NFengguang Wu <fengguang.wu@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: NThomas Meyer <thomas@m3y3r.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add sha224 implementation to sha256_ssse3 module.

This also fixes sha256_ssse3 module autoloading issue when 'sha224' is used
before 'sha256'. Previously in such case, just sha256_generic was loaded and
not sha256_ssse3 (since it did not provide sha224). Now if 'sha256' was used
after 'sha224' usage, sha256_ssse3 would remain unloaded.

Cc: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add sha384 implementation to sha512_ssse3 module.

This also fixes sha512_ssse3 module autoloading issue when 'sha384' is used
before 'sha512'. Previously in such case, just sha512_generic was loaded and
not sha512_ssse3 (since it did not provide sha384). Now if 'sha512' was used
after 'sha384' usage, sha512_ssse3 would remain unloaded. For example, this
happens with tcrypt testing module since it tests 'sha384' before 'sha512'.

Cc: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

'sha512_generic' should set driver name now that there is alternative sha512
provider (sha512_ssse3).
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

free_irq() expects the same pointer that was passed to request_irq(), otherwise
the IRQ is not freed.

The issue was found using the following coccinelle script:

<smpl>
@r1@
type T;
T devid;
@@
request_irq(..., devid)

@r2@
type r1.T;
T devid;
position p;
@@
free_irq@p(..., devid)

@@
position p != r2.p;
@@
*free_irq@p(...)
</smpl>
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch enables the DCP crypto functionality on imx28.
Currently, only aes-128-cbc is supported.
Moreover, the dcpboot misc-device, which is used by Freescale's
SDK tools and uses a non-software-readable OTP-key, is added.

Changes of v2:
- ring buffer for hardware-descriptors
- use of ablkcipher walk
- OTP key encryption/decryption via misc-device
  (compatible to Freescale-SDK)
- overall cleanup

The DCP is also capable of sha1/sha256 but I won't be able to add
that anytime soon.
Tested with built-in runtime-self-test, tcrypt and openssl via
cryptodev 1.6 on imx28-evk and a custom built imx28-board.
Signed-off-by: NTobias Rauter <tobias.rauter@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add Class Context SRC / DEST flags for the LOAD & STORE commands
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add define for "Adjust Output Frame Length" in order to
set the AOFL bit in the IPsec ESP Decapsulation PDB.
Signed-off-by: NAnca-Jeanina Floarea <anca.floarea@freescale.com>
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

add defines for:
    append load immediate command
    setting SEQ LIODN equal to the Non-SEQ LIODN for the job
    replace job descriptor command
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Store command has options to overwrite the Job Desc, Shared Desc or
the entire Descriptor in memory, using the address from
which the Descriptor was fetched.
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

added all supported math funtion on 8 byte boundary with
immediate flag bit set automatically

added MATH_SRC0_DPOVRD & MATH_SRC1_DPOVRD

The function/defines above are needed for creating descriptors
longer than 64 words
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Perform 32-bit left shift of DEST and concatenate with
left 32 bits of SRC1. {DEST[31:0],SRC1[63:32]}
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Acked-by: NMihai Serb <mihai.serb@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In case Store command is used with overwrite Shared Descriptor
feature there is no need for pointer, it is using the
address from which the Shared Descriptor was fetched.
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

SEQ IN PTR command does not require pointer if RTO or PRE bit is set
Updated desc_constr.h accordingly.
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The _XFER stack element size was set too small, 8 bytes, when it needs to be
16 bytes. As _XFER is the last stack element used by these implementations,
the 16 byte stores with 'movdqa' corrupt the stack where the value of register
%r12 is temporarily stored. As these implementations align the stack pointer
to 16 bytes, this corruption did not happen every time.

Patch corrects this issue.
Reported-by: NJulian Wollrath <jwollrath@web.de>
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@iki.fi>
Tested-by: NJulian Wollrath <jwollrath@web.de>
Acked-by: NTim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The Kconfig symbol EXPERIMENTAL was removed in v3.9. So this dependency
makes it impossible to set CRYPTO_DEV_SAHARA. It's unlikely that this is
what is intended, so let's remove this dependency.
Signed-off-by: NPaul Bolle <pebolle@tiscali.nl>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The driver core clears the driver data to NULL after device_release
or on probe failure, since commit 0998d063
(device-core: Ensure drvdata = NULL when no driver is bound).
Thus, it is not needed to manually clear the device driver data to NULL.
Signed-off-by: NJingoo Han <jg1.han@samsung.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The driver core clears the driver data to NULL after device_release
or on probe failure, since commit 0998d063
(device-core: Ensure drvdata = NULL when no driver is bound).
Thus, it is not needed to manually clear the device driver data to NULL.
Signed-off-by: NJingoo Han <jg1.han@samsung.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>