The protocol selection code is now consolidated in a few consecutive
short functions in a single file and is table driven.  Protocol-specific
constraints that influence negotiation are moved into the flags
field of the method structure.  The same protocol version constraints
are now applied in all code paths.  It is now much easier to add
new protocol versions without reworking the protocol selection
logic.

In the presence of "holes" in the list of enabled client protocols
we no longer select client protocols below the hole based on a
subset of the constraints and then fail shortly after when it is
found that these don't meet the remaining constraints (suiteb, FIPS,
security level, ...).  Ideally, with the new min/max controls users
will be less likely to create "holes" in the first place.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

Remove all fixed DH ciphersuites and associated logic.
Reviewed-by: NMatt Caswell <matt@openssl.org>

Various updates following feedback from the recent commit of the new
GOST2012 code.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

This patch contains the necessary changes to provide GOST 2012
ciphersuites in TLS. It requires the use of an external GOST 2012 engine.
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>

We were setting |s->renegotiate| and |s->new_session| to 0 twice in
tls_finish_handshake. This is redundant so now we just do it once!
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

We finish the handshake when we move into the TLS_ST_OK state. At various
points we were also unnecessarily finishing it when we were reading/writing
the Finished message. It's much simpler just to do it in TLS_ST_OK, so
remove the other calls.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

|tls_process_finished| was checking that |peer_finish_md_len| was
non-negative. However neither |tls1_final_finish_mac| or
|ssl3_final_finish_mac| can ever return a negative value, so the check is
superfluous.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Various enums were introduced as part of the state machine rewrite. As a
matter of style it is preferred for these to be typedefs.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Change various state machine functions to use the prefix ossl_statem
instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Move some function definitions around within the state machine to make sure
they are in the correct files. Also create a statem_locl.h header for stuff
entirely local to the state machine code and move various definitions into
it.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Pull out the state machine into a separate sub directory. Also moved some
functions which were nothing to do with the state machine but were in state
machine files. Pulled all the SSL_METHOD definitions into one place...most
of those files had very little left in them any more.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Previously each message specific process function would create its own
PACKET structure. Rather than duplicate all of this code lots of times we
should create it in the state machine itself.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The SSL structure contained a "type" variable that was set to either
SSL_ST_ACCEPT or SSL_ST_CONNECT depending on whether we are the server or
the client. This duplicates the capability of the "server" variable and was
actually rarely used.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Clean up and remove lots of code that is now no longer needed due to the
move to the new state machine.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Move all DTLS client side processing into the new state machine code. A
subsequent commit will clean up the old dead code.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

This swaps the implementation of the client TLS state machine to use the
new state machine code instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The new state machine code will split up the reading and writing of
hanshake messages into discrete phases. In order to facilitate that the
existing "get" type functions will be split into two halves: one to get
the message and one to process it. The "send" type functions will also have
all work relating to constructing the message split out into a separate
function just for that. For some functions there will also be separate
pre and post "work" phases to prepare or update state.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

This is the first drop of the new state machine code.

The rewrite has the following objectives:
- Remove duplication of state code between client and server
- Remove duplication of state code between TLS and DTLS
- Simplify transitions and bring the logic together in a single location
  so that it is easier to validate
- Remove duplication of code between each of the message handling functions
- Receive a message first and then work out whether that is a valid
  transition - not the other way around (the other way causes lots of issues
  where we are expecting one type of message next but actually get something
  else)
- Separate message flow state from handshake state (in order to better
  understand each)
  - message flow state = when to flush buffers; handling restarts in the
    event of NBIO events; handling the common flow of steps for reading a
    message and the common flow of steps for writing a message etc
  - handshake state = what handshake message are we working on now
- Control complexity: only the state machine can change state: keep all
  the state changes local to a file

This builds on previous state machine related work:
- Surface CCS processing in the state machine
- Version negotiation rewrite
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The function ssl3_get_message gets a whole message from the underlying bio
and returns it to the state machine code. The new state machine code will
split this into two discrete steps: get the message header and get the
message body. This commit splits the existing function into these two
sub steps to facilitate the state machine implementation.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

This was obsolete in 2001.  This is not the same as Gost94 digest.
Thanks to Dmitry Belyavsky <beldmit@gmail.com> for review and advice.
Reviewed-by: NMatt Caswell <matt@openssl.org>

Continuing on from the previous commit this moves the processing of DTLS
CCS messages out of the record layer and into the state machine.
Reviewed-by: NTim Hudson <tjh@openssl.org>

The handling of incoming CCS records is a little strange. Since CCS is not
a handshake message it is handled differently to normal handshake messages.
Unfortunately whilst technically it is not a handhshake message the reality
is that it must be processed in accordance with the state of the handshake.
Currently CCS records are processed entirely within the record layer. In
order to ensure that it is handled in accordance with the handshake state
a flag is used to indicate that it is an acceptable time to receive a CCS.

Previously this flag did not exist (see CVE-2014-0224), but the flag should
only really be considered a workaround for the problem that CCS is not
visible to the state machine.

Outgoing CCS messages are already handled within the state machine.

This patch makes CCS visible to the TLS state machine. A separate commit
will handle DTLS.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Miscellaneous updates following review comments on the version negotiation
rewrite patches.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

This commit changes the way that we do server side protocol version
negotiation. Previously we had a whole set of code that had an "up front"
state machine dedicated to the negotiating the protocol version. This adds
significant complexity to the state machine. Historically the justification
for doing this was the support of SSLv2 which works quite differently to
SSLv3+. However, we have now removed support for SSLv2 so there is little
reason to maintain this complexity.

The one slight difficulty is that, although we no longer support SSLv2, we
do still support an SSLv3+ ClientHello in an SSLv2 backward compatible
ClientHello format. This is generally only used by legacy clients. This
commit adds support within the SSLv3 code for these legacy format
ClientHellos.

Server side version negotiation now works in much the same was as DTLS,
i.e. we introduce the concept of TLS_ANY_VERSION. If s->version is set to
that then when a ClientHello is received it will work out the most
appropriate version to respond with. Also, SSLv23_method and
SSLv23_server_method have been replaced with TLS_method and
TLS_server_method respectively. The old SSLv23* names still exist as
macros pointing at the new name, although they are deprecated.

Subsequent commits will look at client side version negotiation, as well of
removal of the old s23* code.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

The return value is checked for 0. This is currently safe but we should
really check for <= 0 since -1 is frequently used for error conditions.
Thanks to Kevin Wojtysiak (Int3 Solutions) and Paramjot Oberoi (Int3
Solutions) for reporting this issue.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Change ssl_set_handshake_header from return void to returning int, and
handle error return code appropriately.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Remove OPENSSL_NO_BUF_FREELISTS. This was turned on by default,
so the work here is removing the 'maintain our own freelist' code.
Also removed a minor old Windows-multibyte/widechar conversion flag.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

Sometimes it fails to format them very well, and sometimes it corrupts them!
This commit moves some particularly problematic ones.
Reviewed-by: NTim Hudson <tjh@openssl.org>

This last one for this ticket.  Removes WIN16.
So long, MS_CALLBACK and MS_FAR.  We won't miss you.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

MS Server gated cryptography is obsolete and dates from the time of export
restrictions on strong encryption and is only used by ancient versions of
MSIE.
Reviewed-by: NMatt Caswell <matt@openssl.org>

indent will not alter them when reformatting comments
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

Previously, state variant was not advanced, which resulted in state
being stuck in the st1 variant (usually "_A").

This broke certificate callback retry logic when accepting connections
that were using SSLv2 ClientHello (hence reusing the message), because
their state never advanced to SSL3_ST_SR_CLNT_HELLO_C variant required
for the retry code path.

Reported by Yichun Zhang (agentzh).
Signed-off-by: NPiotr Sikora <piotr@cloudflare.com>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

Security callback: selects which parameters are permitted including
sensible defaults based on bits of security.

The "parameters" which can be selected include: ciphersuites,
curves, key sizes, certificate signature algorithms, supported
signature algorithms, DH parameters, SSL/TLS version, session tickets
and compression.

In some cases prohibiting the use of a parameters will mean they are
not advertised to the peer: for example cipher suites and ECC curves.
In other cases it will abort the handshake: e.g DH parameters or the
peer key size.

Documentation to follow...