Enabling extended addressing in the h/w requires we always assign the
extended address component (eptr) of the talitos h/w pointer.  This is
for e500 based platforms with large memories.
Signed-off-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

align channel access locks onto separate cache lines (for performance
reasons).  This is done by placing per-channel variables into their own
private struct, and using the cacheline_aligned attribute within that
struct.
Signed-off-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

don't do request->src vs. assoc pointer math - it's the same as adding
assoclen and ivsize (just with more effort).
Signed-off-by: NKim Phillips <kim.phillips@freescale.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This adds support for Marvell's Cryptographic Engines and Security
Accelerator (CESA) which can be found on a few SoC.
Tested with dm-crypt.
Acked-by: NNicolas Pitre <nico@marvell.com>
Signed-off-by: NSebastian Andrzej Siewior <sebastian@breakpoint.cc>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

cryptd_alloc_ahash() will allocate a cryptd-ed ahash for specified
algorithm name. The new allocated one is guaranteed to be cryptd-ed
ahash, so the shash underlying can be gotten via cryptd_ahash_child().
Signed-off-by: NHuang Ying <ying.huang@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Remove the dedicated GHASH implementation in GCM, and uses the GHASH
digest algorithm instead. This will make GCM uses hardware accelerated
GHASH implementation automatically if available.

ahash instead of shash interface is used, because some hardware
accelerated GHASH implementation needs asynchronous interface.
Signed-off-by: NHuang Ying <ying.huang@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

GHASH is implemented as a shash algorithm. The actual implementation
is copied from gcm.c. This makes it possible to add
architecture/hardware accelerated GHASH implementation.
Signed-off-by: NHuang Ying <ying.huang@intel.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch converts authenc to the new ahash interface.
Signed-off-by: NSteffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The aligned ctx helper was using a bogus alignment value thas was
one off the correct value.  Fortunately the current users do not
require anything beyond the natural alignment of the platform so
this hasn't caused a problem.

This patch fixes that and also removes the unnecessary minimum
check since if the alignment is less than the natural alignment
then the subsequent ALIGN operation should be a noop.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch uses crypto_shash_export/crypto_shash_import to prehash
ipad/opad to speed up hmac.  This is partly based on a similar patch
by Steffen Klassert.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

It's undefined behaviour in C to write outside the bounds of an array.
The key expansion routine takes a shortcut of creating 8 words at a
time, but this creates 4 additional words which don't fit in the array.

As everyone is hopefully now aware, GCC is at liberty to make any
assumptions and optimisations it likes in situations where it can
detect that UB has occured, up to and including nasal demons, and
as the indices being accessed in the array are trivially calculable,
it's rash to invite gcc to do take any liberties at all.
Signed-off-by: NPhil Carmody <ext-phil.2.carmody@nokia.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

crypto_init_shash_ops_async() tests for setkey and not for import
before exporting the algorithms import function to ahash.
This patch fixes this.
Signed-off-by: NSteffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

ahash_op_unaligned() and ahash_def_finup() allocate memory atomically,
regardless whether the request can sleep or not. This patch changes
this to use GFP_KERNEL if the request can sleep.
Signed-off-by: NSteffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch provides a default export/import function for all
shash algorithms.  It simply copies the descriptor context as
is done by sha1_generic.

This in essence means that all existing shash algorithms now
support export/import.  This is something that will be depended
upon in implementations such as hmac.  Therefore all new shash
and ahash implementations must support export/import.

For those that cannot obtain a partial result, padlock-sha's
fallback model should be used so that a partial result is always
available.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

    
This patch adds export/import support to sha512-s390 (which includes
sha384-s390).  The exported type is defined by struct sha512_state,
which is basically the entire descriptor state of sha512_generic.

Since sha512-s390 only supports a 64-bit byte count the import
function will reject anything that exceeds that.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch replaces the 32-bit counters in sha512_generic with
64-bit counters.  It also switches the bit count to the simpler
byte count.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch renames struct sha512_ctx and exports it as struct
sha512_state so that other sha512 implementations can use it
as the reference structure for exporting their state.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Although xcbc was converted to shash, it didn't obey the new
requirement that all hash state must be stored in the descriptor
rather than the transform.

This patch fixes this issue and also optimises away the rekeying
by precomputing K2 and K3 within setkey.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch replaces the local xor function with the generic
crypto_xor function.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the finup/export/import functions to the cryptd
ahash implementation.  We simply invoke the underlying shash
operations.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Use struct s390_sha_ctx instead of sha1/sha256_state struct to fix
s390 crypto build break.
Signed-off-by: NSachin Sant <sachinp@in.ibm.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When we encounter partial blocks in finup, we'll invoke the xsha
instruction with a bogus count that is not a multiple of the block
size.  This patch fixes it.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When shash_ahash_finup encounters a null request, we end up not
calling the underlying final function.  This patch fixes that.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When an shash algorithm is exported as ahash, ahash will access
its digest size through hash_alg_common.  That's why the shash
layout needs to match hash_alg_common.  This wasn't the case
because the alignment weren't identical.

This patch fixes the problem.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When the alignment check was made unconditional for ahash we
may end up crashing on shash algorithms because we're always
calling alg->setkey instead of tfm->setkey.

This patch fixes it.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The previous change to allow hashing from states other than the
initial broke compilation on i386 because the inline assembly
tried to squeeze a u64 into a 32-bit register.  As we've already
checked for 32-bit overflows we can simply truncate it to u32,
or unsigned long so that we don't truncate at all on x86-64.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

If shash_alloc_instance() fails, we return the wrong error value.
This patch fixes it.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

If shash_alloc_instance() fails, we return the wrong error value.
This patch fixes it.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

If cryptd_alloc_instance() fails, the return value is uninitialized.
This patch fixes this by setting the return value.
Signed-off-by: NSteffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The crypto4xx SHA implementation keeps the hash state in the tfm
data structure.  This breaks a fundamental requirement of ahash
implementations that they must be reentrant.

This patch disables the broken implementation.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch exports the finup operation where available and adds
a default finup operation for ahash.  The operations final, finup
and digest also will now deal with unaligned result pointers by
copying it.  Finally export/import operations are will now be
exported too.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

We currently use GFP_ATOMIC in the unaligned setkey function
to allocate the temporary aligned buffer.  Since setkey must
be called in a sleepable context, we can use GFP_KERNEL instead.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

When we encounter an unaligned pointer we are supposed to copy
it to a temporary aligned location.  However the temporary buffer
isn't aligned properly.  This patch fixes that.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Some unaligned buffers on the stack weren't zapped properly which
may cause secret data to be leaked.  This patch fixes them by doing
a zero memset.

It is also possible for us to place random kernel stack contents
in the digest buffer if a digest operation fails.  This is fixed
by only copying if the operation succeeded.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Now that all ahash implementations have been converted to the new
ahash type, we can remove old_ahash_alg and its associated support.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch changes crypto4xx to use the new style ahash type.
In particular, we now use ahash_alg to define ahash algorithms
instead of crypto_alg.

This is achieved by introducing a union that encapsulates the
new type and the existing crypto_alg structure.  They're told
apart through a u32 field containing the type value.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch changes cryptd to use the new style ahash type.  In
particular, the instance is enlarged to encapsulate the new
ahash_alg structure.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch changes cryptd to use the template->create function
instead of alloc in anticipation for the switch to new style
ahash algorithms.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the helpers crypto_drop_ahash and crypto_drop_shash
so that these spawns can be dropped without ugly casts.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds a mask parameter to complement the existing type
parameter.  This is useful when instantiating algorithms that
require a mask other than the default, e.g., ahash algorithms.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>