Move some function definitions around within the state machine to make sure
they are in the correct files. Also create a statem_locl.h header for stuff
entirely local to the state machine code and move various definitions into
it.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Pull out the state machine into a separate sub directory. Also moved some
functions which were nothing to do with the state machine but were in state
machine files. Pulled all the SSL_METHOD definitions into one place...most
of those files had very little left in them any more.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Previously each message specific process function would create its own
PACKET structure. Rather than duplicate all of this code lots of times we
should create it in the state machine itself.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The SSL structure contained a "type" variable that was set to either
SSL_ST_ACCEPT or SSL_ST_CONNECT depending on whether we are the server or
the client. This duplicates the capability of the "server" variable and was
actually rarely used.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Clean up and remove lots of code that is now no longer needed due to the
move to the new state machine.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Move all DTLS client side processing into the new state machine code. A
subsequent commit will clean up the old dead code.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

This swaps the implementation of the client TLS state machine to use the
new state machine code instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The new state machine code will split up the reading and writing of
hanshake messages into discrete phases. In order to facilitate that the
existing "get" type functions will be split into two halves: one to get
the message and one to process it. The "send" type functions will also have
all work relating to constructing the message split out into a separate
function just for that. For some functions there will also be separate
pre and post "work" phases to prepare or update state.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

This is the first drop of the new state machine code.

The rewrite has the following objectives:
- Remove duplication of state code between client and server
- Remove duplication of state code between TLS and DTLS
- Simplify transitions and bring the logic together in a single location
  so that it is easier to validate
- Remove duplication of code between each of the message handling functions
- Receive a message first and then work out whether that is a valid
  transition - not the other way around (the other way causes lots of issues
  where we are expecting one type of message next but actually get something
  else)
- Separate message flow state from handshake state (in order to better
  understand each)
  - message flow state = when to flush buffers; handling restarts in the
    event of NBIO events; handling the common flow of steps for reading a
    message and the common flow of steps for writing a message etc
  - handshake state = what handshake message are we working on now
- Control complexity: only the state machine can change state: keep all
  the state changes local to a file

This builds on previous state machine related work:
- Surface CCS processing in the state machine
- Version negotiation rewrite
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The function ssl3_get_message gets a whole message from the underlying bio
and returns it to the state machine code. The new state machine code will
split this into two discrete steps: get the message header and get the
message body. This commit splits the existing function into these two
sub steps to facilitate the state machine implementation.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

This was obsolete in 2001.  This is not the same as Gost94 digest.
Thanks to Dmitry Belyavsky <beldmit@gmail.com> for review and advice.
Reviewed-by: NMatt Caswell <matt@openssl.org>

Continuing on from the previous commit this moves the processing of DTLS
CCS messages out of the record layer and into the state machine.
Reviewed-by: NTim Hudson <tjh@openssl.org>

The handling of incoming CCS records is a little strange. Since CCS is not
a handshake message it is handled differently to normal handshake messages.
Unfortunately whilst technically it is not a handhshake message the reality
is that it must be processed in accordance with the state of the handshake.
Currently CCS records are processed entirely within the record layer. In
order to ensure that it is handled in accordance with the handshake state
a flag is used to indicate that it is an acceptable time to receive a CCS.

Previously this flag did not exist (see CVE-2014-0224), but the flag should
only really be considered a workaround for the problem that CCS is not
visible to the state machine.

Outgoing CCS messages are already handled within the state machine.

This patch makes CCS visible to the TLS state machine. A separate commit
will handle DTLS.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Miscellaneous updates following review comments on the version negotiation
rewrite patches.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

This commit changes the way that we do server side protocol version
negotiation. Previously we had a whole set of code that had an "up front"
state machine dedicated to the negotiating the protocol version. This adds
significant complexity to the state machine. Historically the justification
for doing this was the support of SSLv2 which works quite differently to
SSLv3+. However, we have now removed support for SSLv2 so there is little
reason to maintain this complexity.

The one slight difficulty is that, although we no longer support SSLv2, we
do still support an SSLv3+ ClientHello in an SSLv2 backward compatible
ClientHello format. This is generally only used by legacy clients. This
commit adds support within the SSLv3 code for these legacy format
ClientHellos.

Server side version negotiation now works in much the same was as DTLS,
i.e. we introduce the concept of TLS_ANY_VERSION. If s->version is set to
that then when a ClientHello is received it will work out the most
appropriate version to respond with. Also, SSLv23_method and
SSLv23_server_method have been replaced with TLS_method and
TLS_server_method respectively. The old SSLv23* names still exist as
macros pointing at the new name, although they are deprecated.

Subsequent commits will look at client side version negotiation, as well of
removal of the old s23* code.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

The return value is checked for 0. This is currently safe but we should
really check for <= 0 since -1 is frequently used for error conditions.
Thanks to Kevin Wojtysiak (Int3 Solutions) and Paramjot Oberoi (Int3
Solutions) for reporting this issue.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Change ssl_set_handshake_header from return void to returning int, and
handle error return code appropriately.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Remove OPENSSL_NO_BUF_FREELISTS. This was turned on by default,
so the work here is removing the 'maintain our own freelist' code.
Also removed a minor old Windows-multibyte/widechar conversion flag.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

Sometimes it fails to format them very well, and sometimes it corrupts them!
This commit moves some particularly problematic ones.
Reviewed-by: NTim Hudson <tjh@openssl.org>

This last one for this ticket.  Removes WIN16.
So long, MS_CALLBACK and MS_FAR.  We won't miss you.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

MS Server gated cryptography is obsolete and dates from the time of export
restrictions on strong encryption and is only used by ancient versions of
MSIE.
Reviewed-by: NMatt Caswell <matt@openssl.org>

indent will not alter them when reformatting comments
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

Previously, state variant was not advanced, which resulted in state
being stuck in the st1 variant (usually "_A").

This broke certificate callback retry logic when accepting connections
that were using SSLv2 ClientHello (hence reusing the message), because
their state never advanced to SSL3_ST_SR_CLNT_HELLO_C variant required
for the retry code path.

Reported by Yichun Zhang (agentzh).
Signed-off-by: NPiotr Sikora <piotr@cloudflare.com>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

Security callback: selects which parameters are permitted including
sensible defaults based on bits of security.

The "parameters" which can be selected include: ciphersuites,
curves, key sizes, certificate signature algorithms, supported
signature algorithms, DH parameters, SSL/TLS version, session tickets
and compression.

In some cases prohibiting the use of a parameters will mean they are
not advertised to the peer: for example cipher suites and ECC curves.
In other cases it will abort the handshake: e.g DH parameters or the
peer key size.

Documentation to follow...

(cherry picked from commit 6b42ed4e7104898f4b5b69337589719913b36404)

Partial mitigation of PR#3200
(cherry picked from commit 0294b2be5f4c11e60620c0018674ff0e17b14238)

Revise DTLS code. There was a *lot* of code duplication in the
DTLS code that generates records. This makes it harder to maintain and
sometimes a TLS update is omitted by accident from the DTLS code.

Specifically almost all of the record generation functions have code like
this:

some_pointer = buffer + HANDSHAKE_HEADER_LENGTH;
... Record creation stuff ...
set_handshake_header(ssl, SSL_MT_SOMETHING, message_len);

...

write_handshake_message(ssl);

Where the "Record creation stuff" is identical between SSL/TLS and DTLS or
in some cases has very minor differences.

By adding a few fields to SSL3_ENC to include the header length, some flags
and function pointers for handshake header setting and handshake writing the
code can cope with both cases.

Note: although this passes "make test" and some simple DTLS tests there may
be some minor differences in the DTLS code that have to be accounted for.

This change adds CRYPTO_memcmp, which compares two vectors of bytes in
an amount of time that's independent of their contents. It also changes
several MAC compares in the code to use this over the standard memcmp,
which may leak information about the size of a matching prefix.
(cherry picked from commit 2ee798880a246d648ecddadc5b91367bee4a5d98)

certificate chain instead of an X509 structure.

This makes it easier to enhance code in future and the chain
output functions have access to the CERT_PKEY structure being
used.

New function ssl_add_cert_chain which adds a certificate chain to
SSL internal BUF_MEM. Use this function in ssl3_output_cert_chain
and dtls1_output_cert_chain instead of partly duplicating code.

The cipher definitions of these ciphersuites have been around since SSLeay
but were always disabled. Now OpenSSL supports DH certificates they can be
finally enabled.

Various additional changes were needed to make them work properly: many
unused fixed DH sections of code were untested.