A new method is added to EVP_PKEY_METH as:

    int (*check) (EVP_PKEY_CTX *ctx);

and to EVP_PKEY_ASN1_METHOD as:

    int (*pkey_check) (EVP_PKEY_CTX *ctx);

This is used to check the validity of a specific key.

The order of calls is:
EVP_PKEY_check -> pmeth.check -> ameth.pkey_check.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4337)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4223)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3920)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3301)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2236)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

New function rsa_pss_get_param to extract and sanity check PSS parameters.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Print out RSA-PSS key parameters if present.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

For RSA PSS keys encode and decode parameters when handling public
and private keys.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Split PSS parameter creation. This adds a new function rsa_pss_params_create
which creates PSS parameters from digest and salt values. This will be
used for PSS key generation.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Make RSA method more flexible by using the key type from the
method instead of hard coding EVP_PKEY_RSA: by doing this the
same code supports both RSA and RSA-PSS.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Store hash algorithm used for MGF1 masks in PSS and OAEP modes in PSS and
OAEP parameter structure: this avoids the need to decode part of the ASN.1
structure every time it is used.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2177)

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

GH: #1139

Reviewed-by: NRichard Levitte <levitte@openssl.org>

The rsa_cms_encrypt() function allocates an ASN1_OCTET_STRING but can
then fail to free it in an error condition.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Move rsa_st away from public headers.
Add accessor/writer functions for the public RSA data.
Adapt all other source to use the accessors and writers.
Reviewed-by: NMatt Caswell <matt@openssl.org>

Don't have #error statements in header files, but instead wrap
the contents of that file in #ifndef OPENSSL_NO_xxx
This means it is now always safe to include the header file.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Signed-off-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

This was done by the following
        find . -name '*.[ch]' | /tmp/pl
where /tmp/pl is the following three-line script:
        print unless $. == 1 && m@/\* .*\.[ch] \*/@;
        close ARGV if eof; # Close file to reset $.

And then some hand-editing of other files.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Continuing from previous commit ensure our style is consistent for malloc
return checks.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

Also has changes from from David Woodhouse <David.Woodhouse@intel.com>
and some tweaks from me.
Signed-off-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NTim Hudson <tjh@openssl.org>

There are header files in crypto/ that are used by a number of crypto/
submodules.  Move those to crypto/include/internal and adapt the
affected source code and Makefiles.

The header files that got moved are:

crypto/cryptolib.h
crypto/md32_common.h
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

After the finale, the "real" final part. :)  Do a recursive grep with
"-B1 -w [a-zA-Z0-9_]*_free" to see if any of the preceeding lines are
an "if NULL" check that can be removed.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Don't check for NULL before calling OPENSSL_free
Reviewed-by: NRichard Levitte <levitte@openssl.org>