The return value from the ticket_key callback was not properly handled in
TLSv1.3, so that a ticket was *always* renewed even if the callback
requested that it should not be.

Also the ticket decrypt callback was not being called at all in TLSv1.3.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6198)

When a server call SSL_write_early_data() to write to an unauthenticated
client the buffering BIO is still in place, so we should ensure we flush
the write.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6198)

The TLS code marks records as read when its finished using a record. The DTLS code did
not do that. However SSL_has_pending() relies on it. So we should make DTLS consistent.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6159)

If we have no certificate and we are using "old style" PSKs then we will
always default to using SHA-256 for that PSK. However we may have selected
a ciphersuite that is not based on SHA-256. Therefore if we see that there
are no certificates and we have been configured for "old style" PSKs then
we should prefer SHA-256 based ciphersuites during the selection process.

Fixes #6197
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6215)

Reviewed-by: NRich Salz <rsalz@openssl.org>
GH: #6208

During a full handshake the server is the last one to "speak". The timer
should continue to run until we know that the client has received our last
flight (e.g. because we receive some application data).
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6170)

Otherwise we may end up in a hang when using blocking sockets
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6170)

Fixes #4929
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6170)

The ciphers field in a session contains the stack of ciphers offered by
the client.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6113)

The function SSL_get_shared_ciphers() is supposed to return ciphers shared
by the client and the server. However it only ever returned the client
ciphers.

Fixes #5317
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6113)

Commit d316cdcf introduced some extra
checks into the session-cache update procedure, intended to prevent
the caching of sessions whose resumption would lead to a handshake
failure, since if the server is authenticating the client, there needs to
be an application-set "session id context" to match up to the authentication
context.  While that change is effective for its stated purpose, there
was also some collatoral damage introduced along with the fix -- clients
that set SSL_VERIFY_PEER are not expected to set an sid_ctx, and so
their usage of session caching was erroneously denied.

Fix the scope of the original commit by limiting it to only acting
when the SSL is a server SSL.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5967)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6145)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6094)

Add a script to find them in the future
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/6103)

SSL_kANY, and SSL_aANY were placed in the wrong fields. It makes no
functional difference since these macros evaluate to 0 anyway, which is
the correct value for these fields.

Fixes #6048
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6095)

The MAX_CURVELIST macro defines the total number of in-built SSL/TLS curves
that we support. However it has not been updated as new curves are added.

Fixes #5232
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/6065)

In 1.0.2 and below we always send the same client_version in a reneg
ClientHello that we sent the first time around, regardless of what
version eventually gets negotiated. According to a comment in
statem_clnt.c this is a workaround for some buggy servers that choked if
we changed the version used in the RSA encrypted premaster secret.

In 1.1.0+ this behaviour no longer occurs. This restores the original
behaviour.

Fixes #1651
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6059)

The spec does not prohib certs form using compressed points. It only
requires that points in a key share are uncompressed. It says nothing
about point compression for certs, so we should not fail if a cert uses a
compressed point.

Fixes #5743
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/6055)

This dead code should have been removed as part of #5874 but got missed.

Found by Coverity.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/6049)

In 1.0.2 you could configure automatic ecdh params by using the
ECDHParameters config directive and setting it to the value
"+Automatic" or just "Automatic". This is no longer required in 1.1.0+
but we still recognise the "+Automatic" keyword for backwards compatibility.
However we did not recognise just "Automatic" without the leading "+" which
is equally valid. This commit fixes that omission.

Fixes #4113
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6035)

DTLS was not correctly returning the number of pending bytes left in
a call to SSL_pending(). This makes the detection of truncated packets
almost impossible.

Fixes #5478
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6020)

In 1.1.0 and before calling SSL_in_init() from the info_callback
at SSL_CB_HANDSHAKE_DONE would return 0. This commit fixes it so
that it does again for 1.1.1. This broke Node.

Fixes #4574
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6019)

This will be necessary to enable Wireshark to decrypt QUIC 0-RTT data.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5702)

NSS 3.34 and boringssl have support for "EXPORTER_SECRET"
(https://bugzilla.mozilla.org/show_bug.cgi?id=1287711) which is needed
for QUIC 1-RTT decryption support in Wireshark.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5702)

Fixes #5934
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5983)

If SSL_set_bio() is called with a NULL wbio after a failed connection then
this can trigger an assertion failure. This should be valid behaviour and
the assertion is in fact invalid and can simply be removed.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5966)

The first session ticket sent by the server is actually tacked onto the
end of the first handshake from a state machine perspective. However in
reality this is a post-handshake message, and should be preceeded by a
handshake start event from an info callback perspective.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5874)

Fixes #5721
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5874)

We cannot provide a certificate status on a resumption so we should
ignore this extension in that case.

Fixes #1662
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5896)

./config no-autoload-config
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5959)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5990)

The GOST engine needs to be loaded before we initialise libssl. Otherwise
the GOST ciphersuites are not enabled. However the SSL conf module must
be loaded before we initialise libcrypto. Otherwise we will fail to read
the SSL config from a config file properly.

Another problem is that an application may make use of both libcrypto and
libssl. If it performs libcrypto stuff first and OPENSSL_init_crypto()
is called and loads a config file it will fail if that config file has
any libssl stuff in it.

This commit separates out the loading of the SSL conf module from the
interpretation of its contents. The loading piece doesn't know anything
about SSL so this can be moved to libcrypto. The interpretation of what it
means remains in libssl. This means we can load the SSL conf data before
libssl is there and interpret it when it later becomes available.

Fixes #5809
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5818)

Configuration of TLSv1.3 ciphersuites wasn't working in some cases.

Fixes #5740
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5855)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5799)

Almost all *alloc failures now set an error code.
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5842)

Reviewed-by: NDr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: NRich Salz <rsalz@openssl.org>

Fixes: #4641
GH: #4665

We have been unable to trace the contributor of that code to gain their
agreement for the licence change so the code has to be removed.

This commit reverts that contribution. The contribution had no functional
impact so the original way of doing things is still valid. However the
surrounding code has changed significantly so that the exact code as it
was orignally cannot be used. This commit uses the original code as a basis,
but rewrites it to use the PACKET API.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5808)

We should use the old EVP_PKEY_new_mac_key() instead.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5808)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5808)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5780)