Fixes #5076

Since do_rand_drbg_init() allocates three locks, it needs to ensure
that OPENSSL_init_crypto() is called, otherwise these resources are
not cleaned up properly.
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5083)

Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5038)

The DRGB concept described in NIST SP 800-90A provides for having different
algorithms to generate random output. In fact, the FIPS object module used to
implement three of them, CTR DRBG, HASH DRBG and HMAC DRBG.

When the FIPS code was ported to master in #4019, two of the three algorithms
were dropped, and together with those the entire code that made RAND_DRBG
generic was removed, since only one concrete implementation was left.

This commit restores the original generic implementation of the DRBG, making it
possible again to add additional implementations using different algorithms
(like RAND_DRBG_CHACHA20) in the future.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4998)

The generic part of the FIPS DRBG was implemented in fips_drbg_lib.c and the
algorithm specific parts in fips_drbg_<alg>.c for <alg> in {ctr, hash, hmac}.
Additionally, there was the module fips_drbg_rand.c which contained 'gluing'
code between the RAND_METHOD api and the FIPS DRBG.

When the FIPS code was ported to master in #4019, for some reason the ctr-drbg
implementation from fips_drbg_ctr.c ended up in drbg_rand.c instead of drbg_ctr.c.

This commit renames the module drbg_rand.c back to drbg_ctr.c, thereby restoring
a simple relationship between the original fips modules and the drbg modules
in master:

 fips_drbg_lib.c    =>  drbg_lib.c    /* generic part of implementation */
 fips_drbg_<alg>.c  =>  drbg_<alg>.c  /* algorithm specific implementations */
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4998)

Previously, the RAND_DRBG_uninstantiate() call was not exactly inverse to
RAND_DRBG_instantiate(), because some important member values of the
drbg->ctr member where cleared. Now these values are restored internally.
Signed-off-by: NDr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4402)

Signed-off-by: NDr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4402)

Every DRBG now supports automatic reseeding not only after a given
number of generate requests, but also after a specified time interval.
Signed-off-by: NDr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4402)

A third shared DRBG is added, the so called master DRBG. Its sole purpose
is to reseed the two other shared DRBGs, the public and the private DRBG.
The randomness for the master DRBG is either pulled from the os entropy
sources, or added by the application using the RAND_add() call.

The master DRBG reseeds itself automatically after a given number of generate
requests, but can also be reseeded using RAND_seed() or RAND_add().
A reseeding of the master DRBG is automatically propagated to the public
and private DRBG. This construction fixes the problem, that up to now
the randomness provided by RAND_add() was added only to the public and
not to the private DRBG.
Signed-off-by: NDr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4402)

It's argued that /WX allows to keep better focus on new code, which
motivates its comeback...
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4721)

The drbg's lock must be held across calls to RAND_DRBG_generate()
to prevent simultaneous modification of internal state.

This was observed in practice with simultaneous SSL_new() calls attempting
to seed the (separate) per-SSL RAND_DRBG instances from the global
rand_drbg instance; this eventually led to simultaneous calls to
ctr_BCC_update() attempting to increment drbg->bltmp_pos for their
respective partial final block, violating the invariant that bltmp_pos < 16.
The AES operations performed in ctr_BCC_blocks() makes the race window
quite easy to trigger.  A value of bltmp_pos greater than 16 induces
catastrophic failure in ctr_BCC_final(), with subtraction overflowing
and leading to an attempt to memset() to zero a very large range,
which eventually reaches an unmapped page and segfaults.

Provide the needed locking in get_entropy_from_parent(), as well as
fixing a similar issue in RAND_priv_bytes().  There is also an
unlocked call to RAND_DRBG_generate() in ssl_randbytes(), but the
requisite serialization is already guaranteed by the requirements on
the application's usage of SSL objects, and no further locking is
needed for correct behavior.  In that case, leave a comment noting
the apparent discrepancy and the reason for its safety (at present).
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4328)

The DRBG_RESEED state plays an analogue role to the |reseed_required_flag| in
Appendix B.3.4 of [NIST SP 800-90A Rev. 1]. The latter is a local variable,
the scope of which is limited to the RAND_DRBG_generate() function. Hence there
is no need for a DRBG_RESEED state outside of the generate function. This state
was removed and replaced by a local variable |reseed_required|.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4328)

Reseeding is handled very differently by the classic RAND_METHOD API
and the new RAND_DRBG api. These differences led to some problems when
the new RAND_DRBG was made the default OpenSSL RNG. In particular,
RAND_add() did not work as expected anymore. These issues are discussed
on the thread '[openssl-dev] Plea for a new public OpenSSL RNG API'
and in Pull Request #4328. This commit fixes the mentioned issues,
introducing the following changes:

- Replace the fixed size RAND_BYTES_BUFFER by a new RAND_POOL API which
  facilitates collecting entropy by the get_entropy() callback.
- Don't use RAND_poll()/RAND_add() for collecting entropy from the
  get_entropy() callback anymore. Instead, replace RAND_poll() by
  RAND_POOL_acquire_entropy().
- Add a new function rand_drbg_restart() which tries to get the DRBG
  in an instantiated state by all means, regardless of the current
  state (uninstantiated, error, ...) the DRBG is in. If the caller
  provides entropy or additional input, it will be used for reseeding.
- Restore the original documented behaviour of RAND_add() and RAND_poll()
  (namely to reseed the DRBG immediately) by a new implementation based
  on rand_drbg_restart().
- Add automatic error recovery from temporary failures of the entropy
  source to RAND_DRBG_generate() using the rand_drbg_restart() function.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4328)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4294)

cryptilib.h is the second.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4188)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
GH: #4268

The one creating the DRBG should instantiate it, it's there that we
know which parameters we should use to instantiate it.

This splits the rand init in two parts to avoid a deadlock
because when the global drbg is created it wands to call
rand_add on the global rand method.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
GH: #4268

The DRBG callbacks 'get_entropy()' and 'cleanup_entropy()' are designed
in such a way that the randomness buffer does not have to be allocated
by the calling function. It receives the address of a dynamically
allocated buffer from get_entropy() and returns this address to
cleanup_entropy(), where it is freed. If these two calls are properly
paired, the address can be stored in a stack local variable of the
calling function, so there is no need for having a 'randomness' member
(and a 'filled' member) in 'RAND_DRBG'.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4266)

With the introduction of RAND_poll_ex(), the `RAND_add()` calls were
replaced by meaningless cb(...). This commit changes the 'cb(...)'
calls back to 'rand_add(...)' calls by changing the signature as follows:

-int RAND_poll_ex(RAND_poll_fn cb, void *arg);
+int RAND_poll_ex(RAND_poll_cb rand_add, void *arg);

Changed the function typedef name to 'RAND_poll_cb' to emphasize the fact
that the function type represents a callback function.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4266)

Unlike the NIST DRBG standard, entropy counts are in bits and
buffer lengths are in bytes. This has lead to some confusion and
errors in the past, see my comment on PR 3789.

To clarify the destinction between entropy counts and buffer lengths,
a 'len' suffix has been added to all member names of RAND_DRBG which
represent buffer lengths:

-   {min,max}_{entropy,adin,nonce,pers}
+   {min,max}_{entropy,adin,nonce,pers}len

This change makes naming also more consistent, as can be seen in the
diffs, for example:

-    else if (adinlen > drbg->max_adin) {
+    else if (adinlen > drbg->max_adinlen) {

Also replaced all 'ent's by 'entropy's, following a suggestion of Paul Dale.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4266)

This is a vestige from pre-1.1.0 OpenSSL
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4256)

Don't keep it in the DRBG object, just allocate/free as needed.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4226)

Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4150)

Try to put DRBG and rand_bytes buffers in secure heap
Read the TSC fewer times (but it's still not enabled).
Short-circuit return in win RAND_poll_ex; other minor tweaks and
format-fixes.
Use the _bytes version of rdrand/rdseed
Fix ia32cap checks.
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/4100)

Use atfork to count child forks, and reseed DRBG when the counts don't
match.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4101)

Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4097)

Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4092)

Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/4092)

Add a new global DRBG for private keys used by RAND_priv_bytes.

Add BN_priv_rand() and BN_priv_rand_range() which use RAND_priv_bytes().
Change callers to use the appropriate BN_priv... function.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4076)

Give each SSL object it's own DRBG, chained to the parent global
DRBG which is used only as a source of randomness into the per-SSL
DRBG.  This is used for all session, ticket, and pre-master secret keys.
It is NOT used for ECDH key generation which use only the global
DRBG. (Doing that without changing the API is tricky, if not impossible.)
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4050)

If RAND_add wraps around, XOR with existing. Add test to drbgtest that
does the wrap-around.

Re-order seeding and stop after first success.

Add RAND_poll_ex()

Use the DF and therefore lower RANDOMNESS_NEEDED.  Also, for child DRBG's,
mix in the address as the personalization bits.

Centralize the entropy callbacks, from drbg_lib to rand_lib.
(Conceptually, entropy is part of the enclosing application.)
Thanks to Dr. Matthias St Pierre for the suggestion.

Various code cleanups:
    -Make state an enum; inline RANDerr calls.
    -Add RAND_POLL_RETRIES (thanks Pauli for the idea)
    -Remove most RAND_seed calls from rest of library
    -Rename DRBG_CTX to RAND_DRBG, etc.
    -Move some code from drbg_lib to drbg_rand; drbg_lib is now only the
     implementation of NIST DRBG.
    -Remove blocklength
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4019)

Instead of setting a "magic" global variable to force RAND to keep
consistent state and always generate the same bytestream, have
the fuzzing code install its own RAND_METHOD that does this.  For
BN_RAND_DEBUG, we just don't do it; that debugging was about mucking
with BN's internal representation, not requiring predictable rand
bytes.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4025)

In 'crypto/rand/ossl_rand.c', a call to
'ASYNC_unblock_pause()' is missing in an error case.

CLA: trivial
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4020)

Add a new config param to specify how the CSPRNG should be seeded.
Illegal values or nonsensical combinations (e.g., anything other
than "os" on VMS or HP VOS etc) result in build failures.
Add RDSEED support.
Add RDTSC but leave it disabled for now pending more investigation.

Refactor and reorganization all seeding files (rand_unix/win/vms) so
that they are simpler.

Only require 128 bits of seeding material.

Many document improvements, including why to not use RAND_add() and the
limitations around using load_file/write_file.
Document RAND_poll().

Cleanup Windows RAND_poll and return correct status

More completely initialize the default DRBG.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/3965)

Looking at
http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-90Ar1.pdf
we see that in the CTR_DRBG_Update() algorithm (internal page number 51),
the provided input data is (after truncation to seedlen) xor-d with the
key and V vector (of length keylen and blocklen respectively).  The comment
in ctr_XOR notes that xor-ing with 0 is the identity function, so we can
just ignore the case when the provided input is shorter than seedlen.

The code in ctr_XOR() then proceeds to xor the key with the input, up
to the amount of input present, and computes the remaining input that
could be used to xor with the V vector, before accessing a full 16-byte
stretch of the input vector and ignoring the calculated length.  The correct
behavior is to respect the supplied input length and only xor the
indicated number of bytes.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3971)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3971)

Replacement fix for #3975
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3979)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3974)

As suggested by Kurt.
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/3970)

Ported from the last FIPS release, with DUAL_EC and SHA1 and the
self-tests removed.  Since only AES-CTR is supported, other code
simplifications were done.  Removed the "entropy blocklen" concept.

Moved internal functions to new include/internal/rand.h.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/3789)

Also fix a RANDerr call.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3947)