Glue_helper incorrectly XORs new IV over old IV at end of CBC encryption
function when it should store. This causes CBC encryption to give
incorrect output on multi-page encryption requests.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

cast5/avx incorrectly XORs new IV over old IV at end of CBC encryption
function when it should store. This causes CBC encryption to give
incorrect output on multi-page encryption requests.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add missing tests for ctr(camellia), lrw(camellia), xts(camellia) and ghash,
as these have test vectors available.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Acked-by: NDavid S. Miller <davem@davemloft.net>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Currrently test_aead uses same buffer for destination and source. However
in any places, 'dst != src' take different path than 'dst == src' case.

Therefore make test_aead also run tests with destination buffer being
different than source buffer.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Acked-by: NDavid S. Miller <davem@davemloft.net>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Currrently test_skcipher uses same buffer for destination and source. However
in any places, 'dst != src' take different path than 'dst == src' case.

Therefore make test_skcipher also run tests with destination buffer being
different than source buffer.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Acked-by: NDavid S. Miller <davem@davemloft.net>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

More precisely, test 'long word' and 'long long word' overflow and carry
handling.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Allow non-multi page and multi page skcipher tests to be run on same test template, to avoid
duplicating data.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

we need to configure the TRNG to use more clocks per sample
to handle the two back-to-back 64KiB random descriptor requests
on higher frequency P5040s.
Signed-off-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Ran into this while looking at some new crypto code using FPU
hitting a WARN_ON_ONCE(!irq_fpu_usable()) in the kernel_fpu_begin()
on a x86 kernel that uses the new eagerfpu model. In short, current eagerfpu
changes return 0 for interrupted_kernel_fpu_idle() and the in_interrupt()
thinks it is in the interrupt context because of the local_bh_disable().
Thus resulting in the WARN_ON().

Remove the local_bh_disable/enable() calls around the existing
local_irq_disable/enable() calls. local_irq_disable/enable() already
disables the BH.

 [ If there are any other legitimate users calling kernel_fpu_begin() from
   the process context but with BH disabled, then we can look into fixing the
   irq_fpu_usable() in future. ]
Signed-off-by: NSuresh Siddha <suresh.b.siddha@intel.com>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Convert a nonnegative error return code to a negative one, as returned
elsewhere in the function.

A simplified version of the semantic match that finds this problem is as
follows: (http://coccinelle.lip6.fr/)

// <smpl>
(
if@p1 (\(ret < 0\|ret != 0\))
 { ... return ret; }
|
ret@p1 = 0
)
... when != ret = e1
    when != &ret
*if(...)
{
  ... when != ret = e2
      when forall
 return ret;
}
// </smpl>
Signed-off-by: NPeter Senna Tschudin <peter.senna@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Convert a nonnegative error return code to a negative one, as returned
elsewhere in the function.

A simplified version of the semantic match that finds this problem is as
follows: (http://coccinelle.lip6.fr/)

// <smpl>
(
if@p1 (\(ret < 0\|ret != 0\))
 { ... return ret; }
|
ret@p1 = 0
)
... when != ret = e1
    when != &ret
*if(...)
{
  ... when != ret = e2
      when forall
 return ret;
}
// </smpl>
Signed-off-by: NPeter Senna Tschudin <peter.senna@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Convert a nonnegative error return code to a negative one, as returned
elsewhere in the function.

A simplified version of the semantic match that finds this problem is as
follows: (http://coccinelle.lip6.fr/)

// <smpl>
(
if@p1 (\(ret < 0\|ret != 0\))
 { ... return ret; }
|
ret@p1 = 0
)
... when != ret = e1
    when != &ret
*if(...)
{
  ... when != ret = e2
      when forall
 return ret;
}
// </smpl>
Signed-off-by: NPeter Senna Tschudin <peter.senna@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Convert a nonnegative error return code to a negative one, as returned
elsewhere in the function.

A simplified version of the semantic match that finds this problem is as
follows: (http://coccinelle.lip6.fr/)

// <smpl>
(
if@p1 (\(ret < 0\|ret != 0\))
 { ... return ret; }
|
ret@p1 = 0
)
... when != ret = e1
    when != &ret
*if(...)
{
  ... when != ret = e2
      when forall
 return ret;
}
// </smpl>
Signed-off-by: NPeter Senna Tschudin <peter.senna@gmail.com>
Reviewed-by: NArun Murthy <arunrmurthy83@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

According to SEC v5.0-v5.3 reference manuals.
Signed-off-by: NHoria Geanta <horia.geanta@freescale.com>
Acked-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In current driver, everytime we need to access the rng clock
,ie to enable or disable it, a call to clk_get is done.

This is not correct and the preferred way is to provide a rng data structure
that could be used for accessing rng resources.
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NFabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Adapt clocks to the new i.mx clock framework and fix the following warning:

------------[ cut here ]------------
WARNING: at drivers/clk/clk.c:511 __clk_enable+0x9c/0xac()
Modules linked in:
Backtrace:
[<800124c8>] (dump_backtrace+0x0/0x10c) from [<804172dc>] (dump_stack+0x18/0x1c)
 r7:00000009 r6:000001ff r5:8032cb50 r4:00000000
[<804172c4>] (dump_stack+0x0/0x1c) from [<80021834>] (warn_slowpath_common+0x54)
[<800217e0>] (warn_slowpath_common+0x0/0x6c) from [<80021870>] (warn_slowpath_n)
 r9:80581cac r8:8700a9c0 r7:805ab070 r6:80000013 r5:806133d4
r4:8700a9c0
[<8002184c>] (warn_slowpath_null+0x0/0x2c) from [<8032cb50>] (__clk_enable+0x9c)
[<8032cab4>] (__clk_enable+0x0/0xac) from [<8032cb88>] (clk_enable+0x28/0x44)
 r5:806133d4 r4:8700a9c0
[<8032cb60>] (clk_enable+0x0/0x44) from [<80560f14>] (mxc_rnga_probe+0x68/0x164)
 r7:805ab070 r6:8706ec00 r5:80611314 r4:00000000
[<80560eac>] (mxc_rnga_probe+0x0/0x164) from [<8025914c>] (platform_drv_probe+0)
[<8025912c>] (platform_drv_probe+0x0/0x24) from [<80257c7c>] (driver_probe_devi)
[<80257bfc>] (driver_probe_device+0x0/0x204) from [<80257e94>] (__driver_attach)
 r9:80581cac r8:0000008e r7:00000000 r6:8706ec3c r5:805ab070
r4:8706ec08
[<80257e00>] (__driver_attach+0x0/0x98) from [<8025642c>] (bus_for_each_dev+0x6)
 r7:00000000 r6:80257e00 r5:87035e98 r4:805ab070
[<802563c4>] (bus_for_each_dev+0x0/0x94) from [<80257adc>] (driver_attach+0x20/)
 r7:00000000 r6:873f2380 r5:805ab338 r4:805ab070
[<80257abc>] (driver_attach+0x0/0x28) from [<80256d50>] (bus_add_driver+0x18c/0)
[<80256bc4>] (bus_add_driver+0x0/0x268) from [<802584c4>] (driver_register+0x80)
[<80258444>] (driver_register+0x0/0x134) from [<802594f4>] (platform_driver_reg)
 r7:00000000 r6:805c2e00 r5:00000007 r4:805ab05c
[<802594a8>] (platform_driver_register+0x0/0x60) from [<80259528>] (platform_dr)
[<80259508>] (platform_driver_probe+0x0/0xa4) from [<80560ea0>] (mod_init+0x18/)
 r7:00000000 r6:805c2e00 r5:00000007 r4:87034000
[<80560e88>] (mod_init+0x0/0x24) from [<800086b4>] (do_one_initcall+0x40/0x194)
[<80008674>] (do_one_initcall+0x0/0x194) from [<8053d3f4>] (kernel_init+0xfc/0x)
[<8053d2f8>] (kernel_init+0x0/0x1cc) from [<80027190>] (do_exit+0x0/0x7ec)
---[ end trace 4198eed02050f461 ]---
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NFabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Support for ESNs (extended sequence numbers).
Tested with strongswan by connecting back-to-back P1010RDB with P2020RDB.
Signed-off-by: NHoria Geanta <horia.geanta@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

.cra_list initialization is unneeded and have been removed from all other
crypto modules except 842.

Cc: Robert Jennings <rcj@linux.vnet.ibm.com>
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Acked-by: NSeth Jennings <sjenning@linux.vnet.ibm.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This reverts commit e6ccc727.

Above commit caused performance regression for CAST6. Reverting gives
following increase in tcrypt speed tests (revert-vs-old ratios).

AMD Phenom II X6 1055T, x86-64:

size    ecb             cbc             ctr             lrw             xts
        enc     dec     enc     dec     enc     dec     enc     dec     enc     dec
16b     1.15x   1.17x   1.16x   1.17x   1.16x   1.16x   1.14x   1.19x   1.05x   1.07x
64b     1.19x   1.23x   1.20x   1.22x   1.19x   1.19x   1.16x   1.24x   1.12x   1.12x
256b    1.21x   1.24x   1.22x   1.24x   1.20x   1.20x   1.17x   1.21x   1.16x   1.14x
1kb     1.21x   1.25x   1.22x   1.24x   1.21x   1.21x   1.18x   1.22x   1.17x   1.15x
8kb     1.21x   1.25x   1.22x   1.24x   1.21x   1.21x   1.18x   1.22x   1.18x   1.15x

Cc: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi>
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix "symbol 'x' was not declared. Should it be static?" sparse warnings.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix "symbol 'x' was not declared. Should it be static?" sparse warnings.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix "constant 0xXXXXXXXXXXXXXXXX is so big it's unsigned long" sparse warnings.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix "symbol 'x' was not declared. Should it be static?" sparse warnings.
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Patch replaces 'movb' instructions with 'movzbl' to break false register
dependencies, interleaves instructions better for out-of-order scheduling
and merges constant 16-bit rotation with round-key variable rotation.

tcrypt ECB results:

Intel Core i5-2450M:

size    old-vs-new      new-vs-generic  old-vs-generic
        enc     dec     enc     dec     enc     dec
256     1.13x   1.19x   2.05x   2.17x   1.82x   1.82x
1k      1.18x   1.21x   2.26x   2.33x   1.93x   1.93x
8k      1.19x   1.19x   2.32x   2.33x   1.95x   1.95x

[v2]
 - Do instruction interleaving another way to avoid adding new FPU<=>CPU
   register moves as these cause performance drop on Bulldozer.
 - Improvements to round-key variable rotation handling.
 - Further interleaving improvements for better out-of-order scheduling.

Cc: Johannes Goetzfried <Johannes.Goetzfried@informatik.stud.uni-erlangen.de>
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Patch replaces 'movb' instructions with 'movzbl' to break false register
dependencies, interleaves instructions better for out-of-order scheduling
and merges constant 16-bit rotation with round-key variable rotation.

tcrypt ECB results (128bit key):

Intel Core i5-2450M:

size    old-vs-new      new-vs-generic  old-vs-generic
        enc     dec     enc     dec     enc     dec
256     1.18x   1.18x   2.45x   2.47x   2.08x   2.10x
1k      1.20x   1.20x   2.73x   2.73x   2.28x   2.28x
8k      1.20x   1.19x   2.73x   2.73x   2.28x   2.29x

[v2]
 - Do instruction interleaving another way to avoid adding new FPU<=>CPU
   register moves as these cause performance drop on Bulldozer.
 - Improvements to round-key variable rotation handling.
 - Further interleaving improvements for better out-of-order scheduling.

Cc: Johannes Goetzfried <Johannes.Goetzfried@informatik.stud.uni-erlangen.de>
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Patch replaces 'movb' instructions with 'movzbl' to break false register
dependencies and interleaves instructions better for out-of-order scheduling.

Tested on Intel Core i5-2450M and AMD FX-8100.

tcrypt ECB results:

Intel Core i5-2450M:

size    old-vs-new      new-vs-3way     old-vs-3way
        enc     dec     enc     dec     enc     dec
256     1.12x   1.13x   1.36x   1.37x   1.21x   1.22x
1k      1.14x   1.14x   1.48x   1.49x   1.29x   1.31x
8k      1.14x   1.14x   1.50x   1.52x   1.32x   1.33x

AMD FX-8100:

size    old-vs-new      new-vs-3way     old-vs-3way
        enc     dec     enc     dec     enc     dec
256     1.10x   1.11x   1.01x   1.01x   0.92x   0.91x
1k      1.11x   1.12x   1.08x   1.07x   0.97x   0.96x
8k      1.11x   1.13x   1.10x   1.08x   0.99x   0.97x

[v2]
 - Do instruction interleaving another way to avoid adding new FPU<=>CPU
   register moves as these cause performance drop on Bulldozer.
 - Further interleaving improvements for better out-of-order scheduling.
Tested-by: NBorislav Petkov <bp@alien8.de>
Cc: Johannes Goetzfried <Johannes.Goetzfried@informatik.stud.uni-erlangen.de>
Signed-off-by: NJussi Kivilinna <jussi.kivilinna@mbnet.fi>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

module_pci_driver makes the code simpler by eliminating
module_init and module_exit calls.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

From: Wei Yongjun <yongjun_wei@trendmicro.com.cn>

Remove duplicated include.
Signed-off-by: NWei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

use true/false for bool, fix code alignment, and fix two allocs with
no test.
Signed-off-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: NDevendra Naga <develkernel412222@gmail.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add assembler versions of AES and SHA1 for ARM platforms.  This has provided
up to a 50% improvement in IPsec/TCP throughout for tunnels using AES128/SHA1.

Platform   CPU SPeed    Endian   Before (bps)   After (bps)   Improvement

IXP425      533 MHz      big     11217042        15566294        ~38%
KS8695      166 MHz     little    3828549         5795373        ~51%
Signed-off-by: NDavid McCullough <ucdevel@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add a MAINTAINERS entry for the IBM Power in-Nest Crypto Acceleators
driver.
Signed-off-by: NKent Yoder <key@linux.vnet.ibm.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Support for ESNs (extended sequence numbers).
Tested with strongswan on a P2020RDB back-to-back setup.
Extracted from /etc/ipsec.conf:
esp=aes-sha1-esn-modp4096!
Signed-off-by: NHoria Geanta <horia.geanta@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Generate a link table in case assoc data is a scatterlist.
While at it, add support for handling non-contiguous assoc data and iv.
Signed-off-by: NHoria Geanta <horia.geanta@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

It's more natural to think of these vars as bool rather than int.
Signed-off-by: NHoria Geanta <horia.geanta@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

talitos_edesc_alloc does not need hash_result param.
Checking whether dst scatterlist is NULL or not is all that is required.
Signed-off-by: NHoria Geanta <horia.geanta@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

For IPsec encryption, in the case when:
-the input buffer is fragmented (edesc->src_nents > 0)
-the output buffer is not fragmented (edesc->dst_nents = 0)
the ICV is not output in the link table, but after the encrypted payload.

Copying the ICV must be avoided in this case; consequently the condition
edesc->dma_len > 0 must be more specific, i.e. must depend on the type
of the output buffer - fragmented or not.

Testing was performed by modifying testmgr to support src != dst,
since currently native kernel IPsec does in-place encryption
(src == dst).
Signed-off-by: NHoria Geanta <horia.geanta@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

the entry points and geniv definitions for all aead,
ablkcipher, and hash algorithms are all common; move them to a
single assignment in talitos_alg_alloc().

This assumes it's ok to assign a setkey() on non-hmac algs.
Signed-off-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

lighten driver_algs[] by moving them to talitos_alg_alloc().
Signed-off-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>