Now that the tfm is passed directly to setkey instead of the ctx, we no
longer need to pass the &tfm->crt_flags pointer.

This patch also gets rid of a few unnecessary checks on the key length
for ciphers as the cipher layer guarantees that the key length is within
the bounds specified by the algorithm.

Rather than testing dia_setkey every time, this patch does it only once
during crypto_alloc_tfm.  The redundant check from crypto_digest_setkey
is also removed.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The convention for setkey is that once it is set it should not change,
in particular, init must not wipe out the key set by it.  In fact, init
should always be used after setkey before any digestion is performed.

The only user of crc32c that sets the key is tcrypt.  This patch adds
the necessary init calls there.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Crypto modules should be loadable by their .cra_driver_name, so
we should make MODULE_ALIAS()es with these names. This patch adds
aliases for SHA1 and SHA256 only as that's what we need for
PadLock-SHA driver.
Signed-off-by: NMichal Ludvig <michal@logix.cz>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Spawns lock a specific crypto algorithm in place.  They can then be used
with crypto_spawn_tfm to allocate a tfm for that algorithm.  When the base
algorithm of a spawn is deregistered, all its spawns will be automatically
removed.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch also adds the infrastructure to pick an algorithm based on
their type.  For example, this allows you to select the encryption
algorithm "aes", instead of any algorithm registered under the name
"aes".  For now this is only accessible internally.  Eventually it
will be made available through crypto_alloc_tfm.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The cryptomgr module is a simple manager of crypto algorithm instances.
It ensures that parameterised algorithms of the type tmpl(alg) (e.g.,
cbc(aes)) are always created.

This is meant to satisfy the needs for most users.  For more complex
cases such as deeper combinations or multiple parameters, a netlink
module will be created which allows arbitrary expressions to be parsed
in user-space.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds a notifier chain for algorithm/template registration events.
This will be used to register compound algorithms such as cbc(aes).  In
future this will also be passed onto user-space through netlink.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

A crypto_template generates a crypto_alg object when given a set of
parameters.  this patch adds the basic data structure fo templates
and code to handle their registration/deregistration.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The crypto API is made up of the part facing users such as IPsec and the
low-level part which is used by cryptographic entities such as algorithms.
This patch splits out the latter so that the two APIs are more clearly
delineated.  As a bonus the low-level API can now be modularised if all
algorithms are built as modules.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Up until now we've relied on module reference counting to ensure that the
crypto_alg structures don't disappear from under us.  This was good enough
as long as each crypto_alg came from exactly one module.

However, with parameterised crypto algorithms a crypto_alg object may need
two or more modules to operate.  This means that we need to count the
references to the crypto_alg object directly.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The functions crypto_alg_get and crypto_alg_put operates on the crypto
modules rather than the algorithms.  Therefore it makes sense to call
them crypto_mod_get and crypto_alg_put respectively.

This is needed because we need to have real algorithm reference counters
for parameterised algorithms as they can be unregistered from below by
when their parameter algorithms are themselves unregistered.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The patch passed the trycpt tests and automated filesystem tests.
This rewrite resulted in some nice perfomance increase over my last patch.

Short summary of the tcrypt benchmarks:

Twofish Assembler vs. Twofish C (256bit 8kb block CBC)
encrypt: -27% Cycles
decrypt: -23% Cycles

Twofish Assembler vs. AES Assembler (128bit 8kb block CBC)
encrypt: +18%  Cycles
decrypt: +15% Cycles

Twofish Assembler vs. AES Assembler (256bit 8kb block CBC)
encrypt: -9% Cycles
decrypt: -8% Cycles

Full Output:
http://homepages.tu-darmstadt.de/~fritschi/twofish/tcrypt-speed-twofish-c-x86_64.txt
http://homepages.tu-darmstadt.de/~fritschi/twofish/tcrypt-speed-twofish-asm-x86_64.txt
http://homepages.tu-darmstadt.de/~fritschi/twofish/tcrypt-speed-aes-asm-x86_64.txt


Here is another bonnie++ benchmark with encrypted filesystems. Most runs maxed
out the hd. It should give some idea what the module can do for encrypted filesystem
performance even though you can't see the full numbers.

http://homepages.tu-darmstadt.de/~fritschi/twofish/output_20060610_130806_x86_64.htmlSigned-off-by: NJoachim Fritschi <jfritschi@freenet.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The patch passed the trycpt tests and automated filesystem tests.
This rewrite resulted in some nice perfomance increase over my last patch.

Short summary of the tcrypt benchmarks:

Twofish Assembler vs. Twofish C (256bit 8kb block CBC)
encrypt: -33% Cycles
decrypt: -45% Cycles

Twofish Assembler vs. AES Assembler (128bit 8kb block CBC)
encrypt: +3%  Cycles
decrypt: -22% Cycles

Twofish Assembler vs. AES Assembler (256bit 8kb block CBC)
encrypt: -20% Cycles
decrypt: -36% Cycles

Full Output:
http://homepages.tu-darmstadt.de/~fritschi/twofish/tcrypt-speed-twofish-asm-i586.txt
http://homepages.tu-darmstadt.de/~fritschi/twofish/tcrypt-speed-twofish-c-i586.txt
http://homepages.tu-darmstadt.de/~fritschi/twofish/tcrypt-speed-aes-asm-i586.txt


Here is another bonnie++ benchmark with encrypted filesystems. All runs with
the twofish assembler modules max out the drivespeed. It should give some
idea what the module can do for encrypted filesystem performance even though
you can't see the full numbers.

http://homepages.tu-darmstadt.de/~fritschi/twofish/output_20060611_205432_x86.htmlSigned-off-by: NJoachim Fritschi <jfritschi@freenet.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds a proper driver name and priority to the generic c
implemtation to allow coexistance of c and assembler modules.
Signed-off-by: NJoachim Fritschi <jfritschi@freenet.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch splits up the twofish crypto routine into a common part ( key
setup  ) which will be uses by all twofish crypto modules ( generic-c , i586
assembler and x86_64 assembler ) and generic-c part. It also creates a new
header file which will be used by all 3 modules.

This eliminates all code duplication.

Correctness was verified with the tcrypt module and automated test scripts.
Signed-off-by: NJoachim Fritschi <jfritschi@freenet.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

It makes no sense to build tcrypt into the kernel.  In fact, now that
the driver init function's return status is being checked, it is in
fact harmful to do so.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds speed tests (benchmarks) for digest algorithms.
Tests are run with different buffer sizes (16 bytes, ... 8 kBytes)
and with each buffer multiple tests are run with different update()
sizes (e.g. hash 64 bytes buffer in four 16 byte updates).
There is no correctness checking of the result and all tests and
algorithms use the same input buffer.
Signed-off-by: NMichal Ludvig <michal@logix.cz>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Intentionaly return -EAGAIN from module_init() to ensure
it doesn't stay loaded in the kernel.  The module does all
its work from init() and doesn't offer any runtime
functionality => we don't need it in the memory, do we?
Signed-off-by: NMichal Ludvig <michal@logix.cz>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

We already allow asynchronous removal of existing algorithm modules.  By
allowing the replacement of existing algorithms, we can replace algorithms
without having to wait for for all existing users to complete.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

We do need to change these names now and even more so in future with
instantiated algorithms.  So let's stop lying to the compiler and get
rid of the const modifiers.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds the hooks cra_init/cra_exit which are called during a tfm's
construction and destruction respectively.  This will be used by the instances
to allocate child tfm's.

For now this lets us get rid of the coa_init/coa_exit functions which are
used for exactly that purpose (unlike the dia_init function which is called
for each transaction).

In fact the coa_exit path is currently buggy as it may get called twice
when an error is encountered during initialisation.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Fix a few omissions in passing TFM instead of CTX to algorithms.
Signed-off-by: NMichal Ludvig <michal@logix.cz>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Up until now algorithms have been happy to get a context pointer since
they know everything that's in the tfm already (e.g., alignment, block
size).

However, once we have parameterised algorithms, such information will
be specific to each tfm.  So the algorithm API needs to be changed to
pass the tfm structure instead of the context pointer.

This patch is basically a text substitution.  The only tricky bit is
the assembly routines that need to get the context pointer offset
through asm-offsets.h.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Various digest algorithms operate one block at a time and therefore
keep a temporary buffer of partial blocks.  This buffer does not need
to be initialised since there is a counter which indicates what is and
isn't valid in it.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Some hash modules load/store data words directly.  The digest layer
should pass properly aligned buffer to update()/final() method.  This
patch also add cra_alignmask to some hash modules.
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

On 64-bit platform, reading 64-bit keys (which is supposed to be
32-bit aligned) at a time will result in unaligned access.
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The AES setkey routine writes 64 bytes to the E_KEY area even though
there are only 60 bytes there.  It is in fact safe since E_KEY is
immediately follwed by D_KEY which is initialised afterwards.  However,
doing this may trigger undefined behaviour and makes Coverity unhappy.

So by combining E_KEY and D_KEY into one array we sidestep this issue
altogether.

This problem was reported by Adrian Bunk.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Force 32-bit alignment on keys in tcrypt test vectors.  Also rearrange the
structure to prevent unnecessary padding.
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The "des3_ede" and "serpent" lack cra_alignmask.
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

this patch converts crypto/ to kzalloc usage.
Compile tested with allyesconfig.
Signed-off-by: NEric Sesterhenn <snakebyte@gmx.de>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Since tfm contexts can contain arbitrary types we should provide at least
natural alignment (__attribute__ ((__aligned__))) for them.  In particular,
this is needed on the Xscale which is a 32-bit architecture with a u64 type
that requires 64-bit alignment.  This problem was reported by Ronen Shitrit.

The crypto_tfm structure's size was 44 bytes on 32-bit architectures and
80 bytes on 64-bit architectures.  So adding this requirement only means
that we have to add an extra 4 bytes on 32-bit architectures.

On i386 the natural alignment is 16 bytes which also benefits the VIA
Padlock as it no longer has to manually align its context structure to
128 bits.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Convert open coded rotations to rol32/ror32.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

A bunch of asm/bug.h includes are both not needed (since it will get
pulled anyway) and bogus (since they are done too early).  Removed.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Many cipher implementations use 4-byte/8-byte loads/stores which require
alignment on some architectures.  This patch explicitly sets the alignment
requirements for them.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The cipher code path may allocate up to two blocks of data on the stack.
Therefore we need to place limits on the maximum block size.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

After a partial update, the done pointer is off to the right by 64 bytes.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Since the temporary buffer is used as an argument to cia_decrypt, it must be
aligned by cra_alignmask.  This bug was found by linux@horizon.com.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch avoids shifting the count left and right needlessly for each
call to sha1_update().  It instead can be done only once at the end in
sha1_final().

Keeping the previous test example (sha1_update() successively called with
len=64), a 1.3% performance increase can be observed on i386, or 0.2% on
ARM.  The generated code is also smaller on ARM.
Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch gives more descriptive names to the variables i and j.
Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

The current code unconditionally copy the first block for every call to
sha1_update().  This can be avoided if there is no pending partial block.
This is always the case on the first call to sha1_update() (if the length
is >= 64 of course.

Furthermore, temp does need to be called if sha_transform is never invoked.
Also consolidate the sha_transform calls into one to reduce code size.
Signed-off-by: NNicolas Pitre <nico@cam.org>
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>