Use sg_pcopy_to_buffer() which is better than the function previously used.
Because it doesn't do kmap/kunmap for skipped pages.
Signed-off-by: NAkinobu Mita <akinobu.mita@gmail.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: "James E.J. Bottomley" <JBottomley@parallels.com>
Cc: Douglas Gilbert <dgilbert@interlog.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Horia Geanta <horia.geanta@freescale.com>
Cc: Imre Deak <imre.deak@intel.com>
Cc: Tejun Heo <tj@kernel.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Use dmaengine_submit instead of calling desc->tx_submit manually.
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

Use dmaengine_prep_slave_sg inline function instead of going through the
structures manually.
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

Use dmaengine_device_control inline function instead of going through the
structures manually.
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

Add missing __iomem to struct cryp_register pointers, this solve some
"incorrect type in initializer (different address spaces)" sparse
warnings.
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

Add missing static qualifiers to hash_process_data and hash_hw_final.
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

Always use readl when reading memory mapped registers.
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

The usage of strict_strtoul() is not preferred, because
strict_strtoul() is obsolete. Thus, kstrtoul() should be
used.
Signed-off-by: NJingoo Han <jg1.han@samsung.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

These symbols are referenced only in this file and hence
should be static.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Tested-by: NTobias Rauter <tobiasrauter@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use NULL instead of 0 for pointer variables.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Tested-by: NTobias Rauter <tobiasrauter@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

devm_* APIs are device managed and make cleanup and exit
code simpler.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Tested-by: NTobias Rauter <tobiasrauter@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit 0998d063 (device-core: Ensure drvdata = NULL when no
driver is bound) removes the need to set driver data field to
NULL.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Tested-by: NTobias Rauter <tobiasrauter@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Calling runtime PM API for every block causes serious perf hit to
crypto operations that are done on a long buffer.
As crypto is performed on a page boundary, encrypting large buffers can
cause a series of crypto operations divided by page. The runtime PM API
is also called those many times.

We call runtime_pm_get_sync only at beginning on the session (cra_init)
and runtime_pm_put at the end. This result in upto a 50% speedup as below.
This doesn't make the driver to keep the system awake as runtime get/put
is only called during a crypto session which completes usually quickly.

Before:
root@beagleboard:~# time -v openssl speed -evp aes-128-cbc
Doing aes-128-cbc for 3s on 16 size blocks: 13310 aes-128-cbc's in 0.01s
Doing aes-128-cbc for 3s on 64 size blocks: 13040 aes-128-cbc's in 0.04s
Doing aes-128-cbc for 3s on 256 size blocks: 9134 aes-128-cbc's in 0.03s
Doing aes-128-cbc for 3s on 1024 size blocks: 8939 aes-128-cbc's in 0.01s
Doing aes-128-cbc for 3s on 8192 size blocks: 4299 aes-128-cbc's in 0.00s

After:
root@beagleboard:~# time -v openssl speed -evp aes-128-cbc
Doing aes-128-cbc for 3s on 16 size blocks: 18911 aes-128-cbc's in 0.02s
Doing aes-128-cbc for 3s on 64 size blocks: 18878 aes-128-cbc's in 0.02s
Doing aes-128-cbc for 3s on 256 size blocks: 11878 aes-128-cbc's in 0.10s
Doing aes-128-cbc for 3s on 1024 size blocks: 11538 aes-128-cbc's in 0.05s
Doing aes-128-cbc for 3s on 8192 size blocks: 4857 aes-128-cbc's in 0.03s

While at it, also drop enter and exit pr_debugs, in related code. tracers
can be used for that.

Tested on a Beaglebone (AM335x SoC) board.
Signed-off-by: NJoel A Fernandes <joelagnel@ti.com>
Acked-by: NKevin Hilman <khilman@linaro.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The sahara crypto driver has an incorrect MODULE_DEVICE_TABLE, which
prevents us from actually building this driver as a loadable module.

sahara_dt_ids is a of_device_id array, so we have to use
MODULE_DEVICE_TABLE(of, ...).
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Cc: Javier Martin <javier.martin@vista-silicon.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: NThomas Meyer <thomas@m3y3r.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

free_irq() expects the same pointer that was passed to request_irq(), otherwise
the IRQ is not freed.

The issue was found using the following coccinelle script:

<smpl>
@r1@
type T;
T devid;
@@
request_irq(..., devid)

@r2@
type r1.T;
T devid;
position p;
@@
free_irq@p(..., devid)

@@
position p != r2.p;
@@
*free_irq@p(...)
</smpl>
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch enables the DCP crypto functionality on imx28.
Currently, only aes-128-cbc is supported.
Moreover, the dcpboot misc-device, which is used by Freescale's
SDK tools and uses a non-software-readable OTP-key, is added.

Changes of v2:
- ring buffer for hardware-descriptors
- use of ablkcipher walk
- OTP key encryption/decryption via misc-device
  (compatible to Freescale-SDK)
- overall cleanup

The DCP is also capable of sha1/sha256 but I won't be able to add
that anytime soon.
Tested with built-in runtime-self-test, tcrypt and openssl via
cryptodev 1.6 on imx28-evk and a custom built imx28-board.
Signed-off-by: NTobias Rauter <tobias.rauter@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add Class Context SRC / DEST flags for the LOAD & STORE commands
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Add define for "Adjust Output Frame Length" in order to
set the AOFL bit in the IPsec ESP Decapsulation PDB.
Signed-off-by: NAnca-Jeanina Floarea <anca.floarea@freescale.com>
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

add defines for:
    append load immediate command
    setting SEQ LIODN equal to the Non-SEQ LIODN for the job
    replace job descriptor command
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Store command has options to overwrite the Job Desc, Shared Desc or
the entire Descriptor in memory, using the address from
which the Descriptor was fetched.
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

added all supported math funtion on 8 byte boundary with
immediate flag bit set automatically

added MATH_SRC0_DPOVRD & MATH_SRC1_DPOVRD

The function/defines above are needed for creating descriptors
longer than 64 words
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Perform 32-bit left shift of DEST and concatenate with
left 32 bits of SRC1. {DEST[31:0],SRC1[63:32]}
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Acked-by: NMihai Serb <mihai.serb@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

In case Store command is used with overwrite Shared Descriptor
feature there is no need for pointer, it is using the
address from which the Shared Descriptor was fetched.
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

SEQ IN PTR command does not require pointer if RTO or PRE bit is set
Updated desc_constr.h accordingly.
Signed-off-by: NAndrei Varvara <andrei.varvara@freescale.com>
Reviewed-by: NPhillips Kim-R1AAHA <Kim.Phillips@freescale.com>
Reviewed-by: NFleming Andrew-AFLEMING <AFLEMING@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

By providing an OF match table with a suitable compatible string, we
can ensure the ux500-hasht driver is probed by supplying an associated
DT node in a given platform's Device Tree.

Cc: linux-crypto@vger.kernel.org
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NLee Jones <lee.jones@linaro.org>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

By providing an OF match table with a suitable compatible string, we
can ensure the ux500-crypt driver is probed by supplying an associated
DT node in a given platform's Device Tree.

Cc: linux-crypto@vger.kernel.org
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NLee Jones <lee.jones@linaro.org>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

The Kconfig symbol EXPERIMENTAL was removed in v3.9. So this dependency
makes it impossible to set CRYPTO_DEV_SAHARA. It's unlikely that this is
what is intended, so let's remove this dependency.
Signed-off-by: NPaul Bolle <pebolle@tiscali.nl>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit 0998d063 (device-core: Ensure drvdata = NULL when no
driver is bound) removes the need to set driver data field to
NULL.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Cc: Vladimir Zapolskiy <vzapolskiy@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Commit 0998d063 (device-core: Ensure drvdata = NULL when no
driver is bound) removes the need to set driver data field to
NULL.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Replace calls to deprecated devm_request_and_ioremap by devm_ioremap_resource.

Found with coccicheck and this semantic patch:
 scripts/coccinelle/api/devm_request_and_ioremap.cocci.
Signed-off-by: NLaurent Navet <laurent.navet@gmail.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fixes a race on driver init with registering algorithms where the
driver status flag wasn't being set before self testing started.

  Added the cra_alignmask field for CBC and ECB modes.

  Fixed a bug in GCM where AES block size was being used instead of
authsize.

  Removed use of blkcipher_walk routines for scatterlist processing.
Corner cases in the code prevent us from processing an entire
scatterlist at a time and walking the buffers in block sized chunks
turns out to be unecessary anyway.

  Fixed off-by-one error in saving off extra data in the sha code.

  Fixed accounting error for number of bytes processed in the sha code.
Signed-off-by: NKent Yoder <key@linux.vnet.ibm.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

The Cryp driver is currently silent and the Hash driver prints the
name of its probe function unnecessarily. Let's just put a nice
descriptive one-liner there instead.

Cc: David S. Miller <davem@davemloft.net>
Cc: Andreas Westin <andreas.westin@stericsson.com>
Cc: linux-crypto@vger.kernel.org
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NLee Jones <lee.jones@linaro.org>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

The DMA controller currently takes configuration information from
information passed though dma_channel_request(), but it shouldn't.
Using the API, the DMA channel should only be configured during
a dma_slave_config() call.

Cc: David S. Miller <davem@davemloft.net>
Cc: Andreas Westin <andreas.westin@stericsson.com>
Cc: linux-crypto@vger.kernel.org
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NLee Jones <lee.jones@linaro.org>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

If we fail to prepare the ux500-cryp clock before enabling it the
platform will fail to boot. Here we insure this happens.

Cc: David S. Miller <davem@davemloft.net>
Cc: Andreas Westin <andreas.westin@stericsson.com>
Cc: linux-crypto@vger.kernel.org
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Acked-by: NUlf Hansson <ulf.hansson@linaro.org>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NLee Jones <lee.jones@linaro.org>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

The DMA controller currently takes configuration information from
information passed though dma_channel_request(), but it shouldn't.
Using the API, the DMA channel should only be configured during
a dma_slave_config() call.

Cc: David S. Miller <davem@davemloft.net>
Cc: Andreas Westin <andreas.westin@stericsson.com>
Cc: linux-crypto@vger.kernel.org
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NLee Jones <lee.jones@linaro.org>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

If we fail to prepare the ux500-hash clock before enabling it the
platform will fail to boot. Here we insure this happens.

Cc: David S. Miller <davem@davemloft.net>
Cc: Andreas Westin <andreas.westin@stericsson.com>
Cc: linux-crypto@vger.kernel.org
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NUlf Hansson <ulf.hansson@linaro.org>
Signed-off-by: NLee Jones <lee.jones@linaro.org>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

req->assoc is dma mapped BIDIRECTIONAL and unmapped TO_DEVICE.
Since it is read-only for the device, use TO_DEVICE both for mapping
and unmapping.

Cc: <stable@vger.kernel.org> # 3.9, 3.8
Signed-off-by: NHoria Geanta <horia.geanta@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

For SEC including a RNG block version >= 4, special initialization
must occur before any descriptor that uses RNG block can be
submitted. This initialization is required not only for SEC
with version greater than 5.0, but for SEC with RNG version >=4.
There may be a case where RNG has already been instantiated by
u-boot or boot ROM code.In such SoCs, if RNG is initialized again
SEC would returns "Instantiation error". Hence, the initialization
status of RNG4 should be also checked before doing RNG init.
Signed-off-by: NRuchika Gupta <ruchika.gupta@freescale.com>
Signed-off-by: NAlex Porosanu <alexandru.porosanu@freescale.com>
Signed-off-by: NAndy Fleming <afleming@freescale.com>
Reviewed-by: NVakul Garg <vakul@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The job ring init function creates a platform device for each job ring.
While the job ring is shutdown, e.g. while caam module removal, its
platform device was not being removed. This leads to failure while
reinsertion and then removal of caam module second time.

The following kernel crash dump appears when caam module is reinserted
and then removed again. This patch fixes it.

root@p4080ds:~# rmmod caam.ko
Unable to handle kernel paging request for data at address 0x00000008
Faulting instruction address: 0xf94aca18
Oops: Kernel access of bad area, sig: 11 [#1]
SMP NR_CPUS=8 P4080 DS
Modules linked in: caam(-) qoriq_dbg(O) [last unloaded: caam]
NIP: f94aca18 LR: f94aca18 CTR: c029f950
REGS: eac47d60 TRAP: 0300   Tainted: G           O  (3.8.4-rt2)
MSR: 00029002 <CE,EE,ME>  CR: 22022484  XER: 20000000
DEAR: 00000008, ESR: 00000000
TASK = e49dfaf0[2110] 'rmmod' THREAD: eac46000 CPU: 1
GPR00: f94ad3f4 eac47e10 e49dfaf0 00000000 00000005 ea2ac210 ffffffff 00000000
GPR08: c286de68 e4977ce0 c029b1c0 00000001 c029f950 10029738 00000000 100e0000
GPR16: 00000000 10023d00 1000cbdc 1000cb8c 1000cbb8 00000000 c07dfecc 00000000
GPR24: c07e0000 00000000 1000cbd8 f94e0000 ffffffff 00000000 ea53cd40 00000000
NIP [f94aca18] caam_reset_hw_jr+0x18/0x1c0 [caam]
LR [f94aca18] caam_reset_hw_jr+0x18/0x1c0 [caam]
Call Trace:
[eac47e10] [eac47e30] 0xeac47e30 (unreliable)
[eac47e20] [f94ad3f4] caam_jr_shutdown+0x34/0x220 [caam]
[eac47e60] [f94ac0e4] caam_remove+0x54/0xb0 [caam]
[eac47e80] [c029fb38] __device_release_driver+0x68/0x120
[eac47e90] [c02a05c8] driver_detach+0xd8/0xe0
[eac47eb0] [c029f8e0] bus_remove_driver+0xa0/0x110
[eac47ed0] [c00768e4] sys_delete_module+0x144/0x270
[eac47f40] [c000e2f0] ret_from_syscall+0x0/0x3c
Signed-off-by: NVakul Garg <vakul@freescale.com>
Signed-off-by: NBharat Bhushan <bharat.bhushan@freescale.com>
Reviewed-by: NHoria Geanta <horia.geanta@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>