The drbg's lock must be held across calls to RAND_DRBG_generate()
to prevent simultaneous modification of internal state.

This was observed in practice with simultaneous SSL_new() calls attempting
to seed the (separate) per-SSL RAND_DRBG instances from the global
rand_drbg instance; this eventually led to simultaneous calls to
ctr_BCC_update() attempting to increment drbg->bltmp_pos for their
respective partial final block, violating the invariant that bltmp_pos < 16.
The AES operations performed in ctr_BCC_blocks() makes the race window
quite easy to trigger.  A value of bltmp_pos greater than 16 induces
catastrophic failure in ctr_BCC_final(), with subtraction overflowing
and leading to an attempt to memset() to zero a very large range,
which eventually reaches an unmapped page and segfaults.

Provide the needed locking in get_entropy_from_parent(), as well as
fixing a similar issue in RAND_priv_bytes().  There is also an
unlocked call to RAND_DRBG_generate() in ssl_randbytes(), but the
requisite serialization is already guaranteed by the requirements on
the application's usage of SSL objects, and no further locking is
needed for correct behavior.  In that case, leave a comment noting
the apparent discrepancy and the reason for its safety (at present).
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4328)

The DRBG_RESEED state plays an analogue role to the |reseed_required_flag| in
Appendix B.3.4 of [NIST SP 800-90A Rev. 1]. The latter is a local variable,
the scope of which is limited to the RAND_DRBG_generate() function. Hence there
is no need for a DRBG_RESEED state outside of the generate function. This state
was removed and replaced by a local variable |reseed_required|.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4328)

Reseeding is handled very differently by the classic RAND_METHOD API
and the new RAND_DRBG api. These differences led to some problems when
the new RAND_DRBG was made the default OpenSSL RNG. In particular,
RAND_add() did not work as expected anymore. These issues are discussed
on the thread '[openssl-dev] Plea for a new public OpenSSL RNG API'
and in Pull Request #4328. This commit fixes the mentioned issues,
introducing the following changes:

- Replace the fixed size RAND_BYTES_BUFFER by a new RAND_POOL API which
  facilitates collecting entropy by the get_entropy() callback.
- Don't use RAND_poll()/RAND_add() for collecting entropy from the
  get_entropy() callback anymore. Instead, replace RAND_poll() by
  RAND_POOL_acquire_entropy().
- Add a new function rand_drbg_restart() which tries to get the DRBG
  in an instantiated state by all means, regardless of the current
  state (uninstantiated, error, ...) the DRBG is in. If the caller
  provides entropy or additional input, it will be used for reseeding.
- Restore the original documented behaviour of RAND_add() and RAND_poll()
  (namely to reseed the DRBG immediately) by a new implementation based
  on rand_drbg_restart().
- Add automatic error recovery from temporary failures of the entropy
  source to RAND_DRBG_generate() using the rand_drbg_restart() function.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4328)

An instruction's QUERY function is executed at initialization, iff the required
MSA level is installed. Therefore, it is sufficient to check the bits returned
by the QUERY functions. The MSA level does not have to be checked at every
function call.
crypto/aes/asm/aes-s390x.pl: The AES key schedule must be computed if the
required KM or KMC function codes are not available. Formally, the availability
of a KMC function code does not imply the availability of the corresponding KM
function code.
Signed-off-by: NPatrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4501)

Signed-off-by: NPatrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4501)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4537)

Thanks to David Benjamin for spotting this.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4532)

Reviewed-by: NRich Salz <rsalz@openssl.org>

Few code format fixup
Fix limit computation; was too strict by 2 bytes.
Simplify computation of buffer limits
Checking is strictly same as sizeof(".pem") == 5
Simplify loop of code for certificate filename creation
Fix MAX_PATH usage
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1936)

And also make '-xmpphost' an alias of the '-name' option.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4524)

Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4367)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4509)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4378)

CLA: trivial
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4515)

Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4527)

The previous commit removed version negotiation on an HRR. However we should
still sanity check the contents of the version field.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4527)

Previously if a client received an HRR then we would do version negotiation
immediately - because we know we are going to get TLSv1.3. However this
causes a problem when we emit the 2nd ClientHello because we start changing
a whole load of stuff to ommit things that aren't relevant for < TLSv1.3.
The spec requires that the 2nd ClientHello is the same except for changes
required from the HRR. Therefore the simplest thing to do is to defer the
version negotiation until we receive the ServerHello.

Fixes #4292
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4527)

CLA: trivial
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4529)

Reviewed-by: NBen Kaduk <kaduk@mit.edu>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4497)

Caught by AddressSanitizer
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4497)

Fixes #4444

[extended tests]
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4447)

Names were not removed.
Some comments were updated.
Replace Andy's address with openssl.org
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4516)

Use atomic operations for the counters
Rename malloc_lock to memdbg_lock
Also fix some style errors in mem_dbg.c
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4359)

If q is non-NULL but p is indeed a safe prime, a modified copy
of p could be leaked.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4525)

Test for the bug where early_data is not accepted by the server when it
does not have an SNI callback set up, but the client sent a servername in
the initial ClientHello establishing the session.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4519)

Fixes #4496
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4519)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4485)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4485)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4485)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4485)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4485)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4485)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4485)

Add functions to return DH parameters using NID and to return the
NID if parameters match a named set. Currently this supports only
RFC7919 parameters but could be expanded in future.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4485)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4485)

If BN_FLG_STATIC_DATA is set don't cleanse a->d as it will reside
in read only memory. If BN_FLG_MALLOCED is not set don't modify the
BIGNUM at all.

This change applies to BN_clear_free() and BN_free(). Now the BIGNUM
structure is opaque applications cannot create a BIGNUM structure
without BN_FLG_MALLOCED being set so they are unaffected.

Update internal DH routines so they only copy pointers for read only
parameters.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4485)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4503)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4503)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4503)

Add an ENGINE to EVP_PKEY structure which can be used for cryptographic
operations: this will typically be used by an HSM key to redirect calls
to a custom EVP_PKEY_METHOD.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4503)