This includes a complete rework of how we use TAP::Harness, by adding
a TAP::Parser subclass that allows additional callbacks to be passed
to perform what we need.  The TAP::Parser callbacks we add are:

    ALL         to print all the TAP output to a file (conditionally)
                to collect all the TAP output to an array (conditionally)
    EOF         to print all the collected TAP output (if there is any)
                if any subtest failed

To get TAP output to file, the environment variable HARNESS_TAP_COPY
must be defined, with a file name as value.  That file will be
overwritten unconditionally.

To get TAP output displayed on failure, the make variable VERBOSE_FAILURE
or VF must be defined with a non-emoty value.

Additionally, the output of test recipe names has been changed to only
display its basename.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9862)

... except on VMS, where output from executed programs doesn't seem to be
captured properly by Test::Harness or TAP::Harness.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9862)

Usually, each element in an OSSL_PARAM array will have a unique key.
However, there may be some rare cases when a responder will handle
multiple elements with the same key.  This adds a short passage
explaining this case.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9741)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9741)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9741)

Instead of relying on implicit fetches, try explicit fetches when available.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9121)

Fixes #9287
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9828)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9826)

The module with_fallback.pm was kind of clunky and required a transfer
module.  This change replaces if with a much more generic pragma type
module, which simply appends given directories to @inc (as opposed to
the 'lib' pragma, which prepends the directories to @inc).

This also supports having a file MODULES.txt with sub-directories to
modules.  This ensures that we don't have to spray individual module
paths throughout our perl code, but can have them collected in one
place.

(do note that there is a 'fallback' module on CPAN.  However, it isn't
part of the core perl, and it has no support the any MODULES.txt kind
of construct)
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9826)

It is undefined behaviour to send NULL as either the src, or dest params
in memcpy.

In pkey_kdf.c we had a check to ensure that the src address is non-NULL.
However in some situations it is possible that the dest address could also
be NULL. Specifically in the case where the datalen is 0 and we are using
a newly allocated BUF_MEM.

We add a check of datalen to avoid the undefined behaviour.
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9868)

Make sure we pass the provider side ctx and not the libcrypto side ctx.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9865)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9863)

Also fixes no-dh
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9861)

Fix no-scrypt.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9840)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9832)

Since commit 7c226dfc a chained DRBG does not add additional
data anymore when reseeding from its parent. The reason is that
the size of the additional data exceeded the allowed size when
no derivation function was used.

This commit provides an alternative fix: instead of adding the
entire DRBG's complete state, we just add the DRBG's address
in memory, thereby providing some distinction between the different
DRBG instances.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9832)

Provides a little extra fork-safety on UNIX systems, adding to the
fact that all DRBGs reseed automatically when the fork_id changes.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9832)

When the new OpenSSL CSPRNG was introduced in version 1.1.1,
it was announced in the release notes that it would be fork-safe,
which the old CSPRNG hadn't been.

The fork-safety was implemented using a fork count, which was
incremented by a pthread_atfork handler. Initially, this handler
was enabled by default. Unfortunately, the default behaviour
had to be changed for other reasons in commit b5319bdb, so
the new OpenSSL CSPRNG failed to keep its promise.

This commit restores the fork-safety using a different approach.
It replaces the fork count by a fork id, which coincides with
the process id on UNIX-like operating systems and is zero on other
operating systems. It is used to detect when an automatic reseed
after a fork is necessary.

To prevent a future regression, it also adds a test to verify that
the child reseeds after fork.

CVE-2019-1549
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9832)

There is no deprecated CTRL support for this new field.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9698)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9814)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9814)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9814)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9814)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9814)

This avoids the problems with PBKDF2 and SCRYPT not being of the same form
as the rest.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9814)

Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9805)

Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9805)

Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9805)

Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9805)

Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9805)

Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9805)

Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9805)

Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9805)

An attack is simple, if the first CMS_recipientInfo is valid but the
second CMS_recipientInfo is chosen ciphertext. If the second
recipientInfo decodes to PKCS #1 v1.5 form plaintext, the correct
encryption key will be replaced by garbage, and the message cannot be
decoded, but if the RSA decryption fails, the correct encryption key is
used and the recipient will not notice the attack.

As a work around for this potential attack the length of the decrypted
key must be equal to the cipher default key length, in case the
certifiate is not given and all recipientInfo are tried out.

The old behaviour can be re-enabled in the CMS code by setting the
CMS_DEBUG_DECRYPT flag.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9777)

Due to the dynamic allocation that was added to rand_pool_add_begin
this function could now return a null pointer where it was previously
guaranteed to succeed. But the return value of this function does
not need to be checked by design.

Move rand_pool_grow from rand_pool_add_begin to rand_pool_bytes_needed.
Make an allocation error persistent to avoid falling back to less secure
or blocking entropy sources.

Fixes: a6a66e45 ("Make rand_pool buffers more dynamic in their sizing.")
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9687)

There was a warning about unused variables in this config:
./config --strict-warnings --with-rand-seed=rdcpu
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9687)

Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NNicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/9821)

crypto/rand/rand_win.c(70) : error C2065: 'BCRYPT_USE_SYSTEM_PREFERRED_RNG' : undeclared identifier
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9827)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9753)

An EVP_PKEY can be used for multiple different algorithm operations.
Only one can be used at a time, so we move those into a union.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9753)