Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4596)

Reseeding is handled very differently by the classic RAND_METHOD API
and the new RAND_DRBG api. These differences led to some problems when
the new RAND_DRBG was made the default OpenSSL RNG. In particular,
RAND_add() did not work as expected anymore. These issues are discussed
on the thread '[openssl-dev] Plea for a new public OpenSSL RNG API'
and in Pull Request #4328. This commit fixes the mentioned issues,
introducing the following changes:

- Replace the fixed size RAND_BYTES_BUFFER by a new RAND_POOL API which
  facilitates collecting entropy by the get_entropy() callback.
- Don't use RAND_poll()/RAND_add() for collecting entropy from the
  get_entropy() callback anymore. Instead, replace RAND_poll() by
  RAND_POOL_acquire_entropy().
- Add a new function rand_drbg_restart() which tries to get the DRBG
  in an instantiated state by all means, regardless of the current
  state (uninstantiated, error, ...) the DRBG is in. If the caller
  provides entropy or additional input, it will be used for reseeding.
- Restore the original documented behaviour of RAND_add() and RAND_poll()
  (namely to reseed the DRBG immediately) by a new implementation based
  on rand_drbg_restart().
- Add automatic error recovery from temporary failures of the entropy
  source to RAND_DRBG_generate() using the rand_drbg_restart() function.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4328)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4485)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4503)

When an SSL's context is swtiched from a ticket-enabled context to
a ticket-disabled context in the servername callback, no session-id
is generated, so the session can't be resumed.

If a servername callback changes the SSL_OP_NO_TICKET option, check
to see if it's changed to disable, and whether a session ticket is
expected (i.e. the client indicated ticket support and the SSL had
tickets enabled at the time), and whether we already have a previous
session (i.e. s->hit is set).

In this case, clear the ticket-expected flag, remove any ticket data
and generate a session-id in the session.

If the SSL hit (resumed) and switched to a ticket-disabled context,
assume that the resumption was via session-id, and don't bother to
update the session.

Before this fix, the updated unit-tests in 06-sni-ticket.conf would
fail test #4 (server1 = SNI, server2 = no SNI).
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/1529)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4371)

A new method is added to EVP_PKEY_METH as:

    int (*check) (EVP_PKEY_CTX *ctx);

and to EVP_PKEY_ASN1_METHOD as:

    int (*pkey_check) (EVP_PKEY_CTX *ctx);

This is used to check the validity of a specific key.

The order of calls is:
EVP_PKEY_check -> pmeth.check -> ameth.pkey_check.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4337)

This allows completely arbitrary passphrases to be entered, including
NUL bytes.
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3821)

SNI and ALPN must be set to be consistent with the PSK. Otherwise this is
an error.
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3926)

AEAD cipher mode implementation is based on that used for AES:

  https://tools.ietf.org/html/rfc5116

TLS GCM cipher suites as specified in:

  https://tools.ietf.org/html/rfc6209Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4287)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4137)

Add an interface that allows accessing the scrypt KDF as a PKEY_METHOD.
This fixes #4021 (at least for the scrypt portion of the issue).
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NStephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4026)

Add a new global DRBG for private keys used by RAND_priv_bytes.

Add BN_priv_rand() and BN_priv_rand_range() which use RAND_priv_bytes().
Change callers to use the appropriate BN_priv... function.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4076)

This was previously mistakenly handled as a single error code.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NStephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

Changed HKDF to use EVP_PKEY_CTX_md() (review comment of @snhenson) and
introduced more specific error codes (not only indicating *that* some
parameter is missing, but actually *which* one it is).
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NStephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

HKDF now handles an invalid digest like TLS1-PRF does (i.e., returns
KDF_R_INVALID_DIGEST if the passed digest is not known). Both KDFs now
set the error code KDF_R_UNKNOWN_PARAMETER_TYPE if a type was passed
that is not recognized. This will have the effect of improving debugging
output in case a user uses "openssl pkeyutl -kdf ..." in a wrong way and
result in an actual error code (instead of just "failure" and an empty
error stack).
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NStephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

Introduce KDF_F_PKEY_HKDF_DERIVE and return the KDF_R_MISSING_PARAMETER
error code when required parameters have not been set. This will make
"openssl pkeyutl -kdf HKDF" return a meaningful error message instead of
simply "Public Key operation error".
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NStephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3989)

Ported from the last FIPS release, with DUAL_EC and SHA1 and the
self-tests removed.  Since only AES-CTR is supported, other code
simplifications were done.  Removed the "entropy blocklen" concept.

Moved internal functions to new include/internal/rand.h.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/3789)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3907)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3858)

Document an internal assumption that these are only for use with files,
and return an error if not. That made the code much simpler.
Leave it as writing 1024 bytes, even though we don't need more than 256
from a security perspective.  But the amount isn't specified, now, so we
can change it later if we want.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3864)

Most of the loader function pointers are crucial, they must be defined
unconditionally.  Therefore, let's make sure OSSL_STORE_register_loader
refuses to register incomplete loaders
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3805)

store_attach_pem_bio() creates a STORE_CTX with the 'file' scheme
loader backend in PEM reading mode on an already opened BIO.
store_detach_pem_bio() detaches the STORE_CTX from the BIO and
destroys it (without destroying the BIO).

These two functions can be used in place of STORE_open() and
STORE_close(), and are present as internal support for other OpenSSL
functions.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2745)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3483)

This has it recognised when the given path is a directory.  In that
case, the file loader will give back a series of names, all as URI
formatted as possible given the incoming URI.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3542)

Add a separate handler for encrypted PKCS#8 data.  This uses the new
restart functionality.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3542)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3542)

Some containers might very simply decode into something new that
deserves to be considered as new (embedded) data.  With the help of a
special OSSL_STORE_INFO type, make that new data available to the
loader functions so they can start over.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3542)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3542)

Containers are objects that are containers for a bunch of other
objects with types we recognise but aren't readable in a stream.  Such
containers are read and parsed, and their content is cached, to be
served one object at a time.

This extends the FILE_HANDLER type to include a function to destroy
the cache and a function to simulate the EOF check.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3542)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3542)

This STORE module adds the following functionality:

- A function OSSL_STORE_open(), OSSL_STORE_load() and OSSL_STORE_close()
  that accesses a URI and helps loading the supported objects (PKEYs,
  CERTs and CRLs for the moment) from it.
- An opaque type OSSL_STORE_INFO that holds information on each loaded
  object.
- A few functions to retrieve desired data from a OSSL_STORE_INFO
  reference.
- Functions to register and unregister loaders for different URI
  schemes.  This enables dynamic addition of loaders from applications
  or from engines.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3542)

The BSD cryptodev.h doesn't have things like COP_FLAG_WRITE_IV and
COP_FLAG_UPDATE.  In that case, we need to implement that
functionality ourselves.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3744)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3670)

See old GitHub PR 38.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3714)

Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/3623)

Add "*" as indicator meaning the function/reason is removed, so put an
empty string in the function/reason string table; this preserves backward
compatibility by keeping the #define's.

In state files, trailing backslash means text is on the next line.

Add copyright to state files
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3640)

Run perltidy on util/mkerr
Change some mkerr flags, write some doc comments
Make generated tables "const" when genearting lib-internal ones.
Add "state" file for mkerr
Renerate error tables and headers
Rationalize declaration of ERR_load_XXX_strings
Fix out-of-tree build
Add -static; sort flags/vars for options.
Also tweak code output
Moved engines/afalg to engines (from master)
Use -static flag
Standard engine #include's of errors
Don't linewrap err string tables unless necessary
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3392)