The original intent of SSL_PKEY_RSA_SIGN and SSL_PKEY_RSA_ENC was to
support two different keys for RSA signing and decrypt. However this
was never implemented and we only ever set one key and the other was
always NULL. Replace with single SSL_PKEY_RSA type.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2587)

While it's always safe to read |SSL_MAX_SSL_SESSION_ID_LENGTH| bytes
from an |SSL_SESSION|'s |session_id| array, the hash function would do
so with without considering if all those bytes had been written to.

This change checks |session_id_length| before possibly reading
uninitialised memory. Since the result of the hash function was already
attacker controlled, and since a lookup of a short session ID will
always fail, it doesn't appear that this is anything more than a clean
up.

In particular, |ssl_get_prev_session| uses a stack-allocated placeholder
|SSL_SESSION| as a lookup key, so the |session_id| array may be
uninitialised.

This was originally found with libFuzzer and MSan in
https://boringssl.googlesource.com/boringssl/+/e976e4349d693b4bbb97e1694f45be5a1b22c8c7,
then by Robert Swiecki with honggfuzz and MSan here. Thanks to both.
Reviewed-by: NGeoff Thorpe <geoff@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2583)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2339)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2287)

session_ctx and initial_ctx are aliases of each other, and with the
opaque data structures, there's no need to keep both around. Since
there were more references of session_ctx, replace all instances of
initial_ctx with session_ctx.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2334)

When we have support for KeyUpdate we might consider doing that instead.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2259)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2259)

The record layer was making decisions that should really be left to the
state machine around unexpected handshake messages that are received after
the initial handshake (i.e. renegotiation related messages). This commit
removes that code from the record layer and updates the state machine
accordingly. This simplifies the state machine and paves the way for
handling other messages post-handshake such as the NewSessionTicket in
TLSv1.3.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2259)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2288)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1618)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1646)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2157)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2052)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2052)

The existing code used the magic number -1 to represent the absence of
a status_type in the extension. This commit replaces it with a macro.

Perl changes reviewed by Richard Levitte. Non-perl changes reviewed by Rich
Salz
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>

GH: #1500

Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1554)

This is a skin deep change, which simply renames most places where we talk
about curves in a TLS context to groups. This is because TLS1.3 has renamed
the extension, and it can now include DH groups too. We still only support
curves, but this rename should pave the way for a future extension for DH
groups.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Travis is reporting one file at a time shadowed variable warnings where
"read" has been used. This attempts to go through all of libssl and replace
"read" with "readbytes" to fix all the problems in one go.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

We also modify the SSL_get_error() function to handle the fact that with
SSL_write_ex() the error return is 0 not -1, and fix some bugs in the
SSL BIO reading.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Writing still to be done
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Includes addition of the various options to s_server/s_client. Also adds
one of the new TLS1.3 ciphersuites.

This isn't "real" TLS1.3!! It's identical to TLS1.2 apart from the protocol
and the ciphersuite...and the ciphersuite is just a renamed TLS1.2 one (not
a "real" TLS1.3 ciphersuite).
Reviewed-by: NRich Salz <rsalz@openssl.org>

For convenience, combine getting a new ref for the new SSL_CTX
with assigning the store and freeing the old one.
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1755)

This reverts commit 81f9ce1e.
Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1594)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

Run util/openssl-format-source on ssl/

Some comments and hand-formatted tables were fixed up
manually by disabling auto-formatting.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Changes them to have clearer ownership semantics, as suggested in
https://github.com/openssl/openssl/pull/1372#discussion_r73232196.
Reviewed-by: NEmilia Käsper <emilia@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1408)

Commit d8e8590e ("Fix missing return value checks in SCTP") made the
DTLS handshake fail, even for non-SCTP connections, if
SSL_export_keying_material() fails. Which it does, for DTLS1_BAD_VER.

Apply the trivial fix to make it succeed, since there's no real reason
why it shouldn't even though we never need it.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

SSL_set_rbio() and SSL_set_wbio() are new functions in 1.1.0 and really
should be called SSL_set0_rbio() and SSL_set0_wbio(). The old
implementation was not consistent with what "set0" means though as there
were special cases around what happens if the rbio and wbio are the same.
We were only ever taking one reference on the BIO, and checking everywhere
whether the rbio and wbio are the same so as not to double free.

A better approach is to rename the functions to SSL_set0_rbio() and
SSL_set0_wbio(). If an existing BIO is present it is *always* freed
regardless of whether the rbio and wbio are the same or not. It is
therefore the callers responsibility to ensure that a reference is taken
for *each* usage, i.e. one for the rbio and one for the wbio.

The legacy function SSL_set_bio() takes both the rbio and wbio in one go
and sets them both. We can wrap up the old behaviour in the implementation
of that function, i.e. previously if the rbio and wbio are the same in the
call to this function then the caller only needed to ensure one reference
was passed. This behaviour is retained by internally upping the ref count.

This commit was inspired by BoringSSL commit f715c423224.

RT#4572
Reviewed-by: NRich Salz <rsalz@openssl.org>

When setting the read bio we free up any old existing one. However this can
lead to a double free if the existing one is the same as the write bio.
Reviewed-by: NRich Salz <rsalz@openssl.org>