TLS1.0 and TLS1.1 say you SHOULD ignore unrecognised record types, but
TLS 1.2 says you MUST send an unexpected message alert. We swap to the
TLS 1.2 behaviour for all protocol versions to prevent issues where no
progress is being made and the peer continually sends unrecognised record
types, using up resources processing them.

Issue reported by 郭志攀
Reviewed-by: NTim Hudson <tjh@openssl.org>

The function ssl3_read_n() takes a parameter |clearold| which, if set,
causes any old data in the read buffer to be forgotten, and any unread data
to be moved to the start of the buffer. This is supposed to happen when we
first read the record header.

However, the data move was only taking place if there was not already
sufficient data in the buffer to satisfy the request. If read_ahead is set
then the record header could be in the buffer already from when we read the
preceding record. So with read_ahead we can get into a situation where even
though |clearold| is set, the data does not get moved to the start of the
read buffer when we read the record header. This means there is insufficient
room in the read buffer to consume the rest of the record body, resulting in
an internal error.

This commit moves the |clearold| processing to earlier in ssl3_read_n()
to ensure that it always takes place.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

We add ssl_cipher_get_overhead() as an internal function, to avoid
having too much ciphersuite-specific knowledge in DTLS_get_data_mtu()
itself. It's going to need adjustment for TLSv1.3... but then again, so
is fairly much *all* of the SSL_CIPHER handling. This bit is in the noise.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

When matching a ciphersuite if we are given an id, make sure we use it
otherwise we will match another ciphersuite which is identical except for
the TLS version.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Includes addition of the various options to s_server/s_client. Also adds
one of the new TLS1.3 ciphersuites.

This isn't "real" TLS1.3!! It's identical to TLS1.2 apart from the protocol
and the ciphersuite...and the ciphersuite is just a renamed TLS1.2 one (not
a "real" TLS1.3 ciphersuite).
Reviewed-by: NRich Salz <rsalz@openssl.org>

For convenience, combine getting a new ref for the new SSL_CTX
with assigning the store and freeing the old one.
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1755)

Rename some parameters.
Also change handling of buffer sizes >INT_MAX in length.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Also extend BIO_METHOD to be able to supply an implementation for the new
BIO_write_ex function.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Also extend BIO_METHOD to be able to supply an implementation for the new
BIO_read function.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

A zero return from BIO_read()/BIO_write() could mean that an IO operation
is retryable. A zero return from SSL_read()/SSL_write() means that the
connection has been closed down (either cleanly or not). Therefore we
should not propagate a zero return value from BIO_read()/BIO_write() back
up the stack to SSL_read()/SSL_write(). This could result in a retryable
failure being treated as fatal.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

OpenSSL 1.1.0 will negotiate EtM on DTLS but will then not actually *do* it.

If we use DTLSv1.2 that will hopefully be harmless since we'll tend to use
an AEAD ciphersuite anyway. But if we're using DTLSv1, then we certainly
will end up using CBC, so EtM is relevant — and we fail to interoperate with
anything that implements EtM correctly.

Fixing it in HEAD and 1.1.0c will mean that 1.1.0[ab] are incompatible with
1.1.0c+... for the limited case of non-AEAD ciphers, where they're *already*
incompatible with other implementations due to this bug anyway. That seems
reasonable enough, so let's do it. The only alternative is just to turn it
off for ever... which *still* leaves 1.0.0[ab] failing to communicate with
non-OpenSSL implementations anyway.

Tested against itself as well as against GnuTLS both with and without EtM.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

The prevailing style seems to not have trailing whitespace, but a few
lines do. This is mostly in the perlasm files, but a few C files got
them after the reformat. This is the result of:

  find . -name '*.pl' | xargs sed -E -i '' -e 's/( |'$'\t'')*$//'
  find . -name '*.c' | xargs sed -E -i '' -e 's/( |'$'\t'')*$//'
  find . -name '*.h' | xargs sed -E -i '' -e 's/( |'$'\t'')*$//'

Then bn_prime.h was excluded since this is a generated file.

Note mkerr.pl has some changes in a heredoc for some help output, but
other lines there lack trailing whitespace too.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

We now set the handshake header, and close the packet directly in the
write_state_machine. This is now possible because it is common for all
messages.
Reviewed-by: NRich Salz <rsalz@openssl.org>

tls_construct_finished() used to have different arguments to all of the
other construction functions. It doesn't anymore, so there is no neeed to
treat it as a special case.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Ensure all message types work the same way including CCS so that the state
machine doesn't need to know about special cases. Put all the special logic
into ssl_set_handshake_header() and ssl_close_construct_packet().
Reviewed-by: NRich Salz <rsalz@openssl.org>

Move setting the handshake header up a level into the state machine code
in order to reduce boilerplate.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Instead of initialising, finishing and cleaning up the WPACKET in every
message construction function, we should do it once in
write_state_machine().
Reviewed-by: NRich Salz <rsalz@openssl.org>

ssl_set_handshake_header2() was only ever a temporary name while we had
to have ssl_set_handshake_header() for code that hadn't been converted to
WPACKET yet. No code remains that needed that so we can rename it.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Remove the old ssl_set_handshake_header() implementations. Later we will
rename ssl_set_handshake_header2() to ssl_set_handshake_header().
Reviewed-by: NRich Salz <rsalz@openssl.org>

This is no longer needed now that all messages use WPACKET
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

A convenience macro was using the wrong underlying function.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

A couple of the WPACKET_sub_memcpy* macros were mis-named.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

In plain PSK we don't need to do anymore construction after the preamble.
We weren't detecting this case and treating it as an unknown cipher.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

WPACKET_allocate_bytes() requires you to know the size of the data you
are allocating for, before you create it. Sometimes this isn't the case,
for example we know the maximum size that a signature will be before we
create it, but not the actual size. WPACKET_reserve_bytes() enables us to
reserve bytes in the WPACKET, but not count them as written yet. We then
subsequently need to acall WPACKET_allocate_bytes to actually count them as
written.
Reviewed-by: NRich Salz <rsalz@openssl.org>

This was a temporary function needed during the conversion to WPACKET. All
callers have now been converted to the new way of doing this so this
function is no longer required.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>