Add an interface that allows accessing the scrypt KDF as a PKEY_METHOD.
This fixes #4021 (at least for the scrypt portion of the issue).
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NStephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4026)

Add missing ASN1_TIME functions

Do some cleanup of the ASN1_TIME code.
Add ASN1_TIME_normalize() to normalize ASN1_TIME structures.
Add ASN1_TIME_compare() to compare two ASN1_TIME structures.
Add ASN1_TIME_cmp_time_t() to compare an ASN1_TIME to time_t
(generic version of ASN1_UTCTIME_cmp_time_t()).

Replace '0' .. '9' compares with isdigit()
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2753)

Add a new global DRBG for private keys used by RAND_priv_bytes.

Add BN_priv_rand() and BN_priv_rand_range() which use RAND_priv_bytes().
Change callers to use the appropriate BN_priv... function.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4076)

Give each SSL object it's own DRBG, chained to the parent global
DRBG which is used only as a source of randomness into the per-SSL
DRBG.  This is used for all session, ticket, and pre-master secret keys.
It is NOT used for ECDH key generation which use only the global
DRBG. (Doing that without changing the API is tricky, if not impossible.)
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4050)

If RAND_add wraps around, XOR with existing. Add test to drbgtest that
does the wrap-around.

Re-order seeding and stop after first success.

Add RAND_poll_ex()

Use the DF and therefore lower RANDOMNESS_NEEDED.  Also, for child DRBG's,
mix in the address as the personalization bits.

Centralize the entropy callbacks, from drbg_lib to rand_lib.
(Conceptually, entropy is part of the enclosing application.)
Thanks to Dr. Matthias St Pierre for the suggestion.

Various code cleanups:
    -Make state an enum; inline RANDerr calls.
    -Add RAND_POLL_RETRIES (thanks Pauli for the idea)
    -Remove most RAND_seed calls from rest of library
    -Rename DRBG_CTX to RAND_DRBG, etc.
    -Move some code from drbg_lib to drbg_rand; drbg_lib is now only the
     implementation of NIST DRBG.
    -Remove blocklength
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4019)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4014)

This was previously mistakenly handled as a single error code.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NStephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

Changed HKDF to use EVP_PKEY_CTX_md() (review comment of @snhenson) and
introduced more specific error codes (not only indicating *that* some
parameter is missing, but actually *which* one it is).
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NStephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

HKDF now handles an invalid digest like TLS1-PRF does (i.e., returns
KDF_R_INVALID_DIGEST if the passed digest is not known). Both KDFs now
set the error code KDF_R_UNKNOWN_PARAMETER_TYPE if a type was passed
that is not recognized. This will have the effect of improving debugging
output in case a user uses "openssl pkeyutl -kdf ..." in a wrong way and
result in an actual error code (instead of just "failure" and an empty
error stack).
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NStephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

Introduce KDF_F_PKEY_HKDF_DERIVE and return the KDF_R_MISSING_PARAMETER
error code when required parameters have not been set. This will make
"openssl pkeyutl -kdf HKDF" return a meaningful error message instead of
simply "Public Key operation error".
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NStephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

The names of these ciphers have an "SSL_" prefix, but the RFC names use
"TLS_":
https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-4

This dates back to these ciphers being originally defined in SSLv3. As
SSLv3 is on its way out anyway and this is a new set of APIs,
consistently use the TLS names.
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4007)

Add functions to enumerate public key methods. Add test to ensure table
is in the correct order.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4015)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4044)

Remove references to it in documentation.

Unfortunately, it is too late to renumber symbols in libcrypto.num
and avoid the NOEXIST entry there.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3860)

This symbol was added in commit d33b215b
but was only used in certain (presumed uncommon) preprocessor conditionals,
as no build failures have been reported yet.

Reported by Balaji Marisetti.

Closes: #4029
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4030)

Instead of setting a "magic" global variable to force RAND to keep
consistent state and always generate the same bytestream, have
the fuzzing code install its own RAND_METHOD that does this.  For
BN_RAND_DEBUG, we just don't do it; that debugging was about mucking
with BN's internal representation, not requiring predictable rand
bytes.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4025)

Since scrypt PBKDF can be used both in PKCS#5 and PKCS#12 files,
do share the code between them.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1334)

OpenSSL already has the feature of SSL_MODE_RELEASE_BUFFERS that can
be set to release the read or write buffers when data has finished
reading or writing. OpenSSL will automatically re-allocate the buffers
as needed. This can be quite aggressive in terms of memory allocation.

This provides a manual mechanism. SSL_free_buffers() will free
the data buffers if there's no pending data. SSL_alloc_buffers()
will realloc them; but this function is not strictly necessary, as it's
still done automatically in the state machine.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2240)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3943)

Changes the EC_KEY_METHOD_get_* family to not need a EC_KEY_METHOD* as
its first parameter, but a const EC_KEY_METHOD*, which is entirely
sufficient.
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
GH: #3985

Fixes: issue #3747

make SSL_CIPHER_standard_name globally available and introduce a new
function OPENSSL_cipher_name.

A new option '-convert' is also added to 'openssl ciphers' app.

Documentation and test cases are added.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/3859)

Ported from the last FIPS release, with DUAL_EC and SHA1 and the
self-tests removed.  Since only AES-CTR is supported, other code
simplifications were done.  Removed the "entropy blocklen" concept.

Moved internal functions to new include/internal/rand.h.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/3789)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3907)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3918)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3858)

SSL_OP_ALL was set in 0x0BFF so reusing some of these bits would cause
ABI compatibility issues.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3833)

1.1.0 included the previous value for SSL_OP_ALLOW_NO_DHE_KEX in
SSL_OP_ALL. This might cause binary compatibility issues. We should choose
a value that is not in SSL_OP_ALL.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3833)

Allow that mode to be configured if desired.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3833)

Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3852)

Document an internal assumption that these are only for use with files,
and return an error if not. That made the code much simpler.
Leave it as writing 1024 bytes, even though we don't need more than 256
from a security perspective.  But the amount isn't specified, now, so we
can change it later if we want.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3864)

And BN_pseudo_rand_range is really BN_rand_range.
Document that we might deprecate those functions.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3743)

Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/3643)

Instead, make it possible to disable the console reader that's part of
the UI module.  This makes it possible to use the UI API and other UI
methods in environments where the console reader isn't useful.

To disable the console reader, configure with 'no-ui-console' /
'disable-ui-console'.

'no-ui' / 'disable-ui' is now an alias for  'no-ui-console' /
'disable-ui-console'.

Fixes #3806
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3820)

Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/3815)

Only for Unix platforms
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3754)

Most of the loader function pointers are crucial, they must be defined
unconditionally.  Therefore, let's make sure OSSL_STORE_register_loader
refuses to register incomplete loaders
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3805)

store_attach_pem_bio() creates a STORE_CTX with the 'file' scheme
loader backend in PEM reading mode on an already opened BIO.
store_detach_pem_bio() detaches the STORE_CTX from the BIO and
destroys it (without destroying the BIO).

These two functions can be used in place of STORE_open() and
STORE_close(), and are present as internal support for other OpenSSL
functions.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2745)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3483)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3542)

This has it recognised when the given path is a directory.  In that
case, the file loader will give back a series of names, all as URI
formatted as possible given the incoming URI.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3542)