if we have a malloc |x = OPENSSL_malloc(...)| sometimes we check |x|
for NULL and sometimes we treat it as a boolean |if(!x) ...|. Standardise
the approach in libssl.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

The SSL object was being deref'd and then there was a later redundant check
to see if it is NULL. We assume all SSL_foo functions pass a non NULL SSL
object and do not check it.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>

The SCTP code is not compiled by default. This fixes some compilation
problems in that code.
Reviewed-by: NTim Hudson <tjh@openssl.org>

We were setting |s->renegotiate| and |s->new_session| to 0 twice in
tls_finish_handshake. This is redundant so now we just do it once!
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

We finish the handshake when we move into the TLS_ST_OK state. At various
points we were also unnecessarily finishing it when we were reading/writing
the Finished message. It's much simpler just to do it in TLS_ST_OK, so
remove the other calls.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

Rebuild error source files: the new mkerr.pl functionality will now
pick up and translate static function names properly.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The various dtls1_get*_methods did not handle the DTLS_ANY_VERSION case,
so this needed to be added.
Reviewed-by: NTim Hudson <tjh@openssl.org>

A buggy application that call SSL_write with a different length after a
NBIO event could cause an OPENSSL_assert to be reached. The assert is not
actually necessary because there was an explicit check a little further
down that would catch this scenario. Therefore remove the assert an move
the check a little higher up.
Reviewed-by: NRich Salz <rsalz@openssl.org>

An OPENSSL_assert was being used which could fail (e.g. on a malloc
failure).
Reviewed-by: NRich Salz <rsalz@openssl.org>

This OPENSSL_assert in (d)tls1_hearbeat is trivially always going to be
true because it is testing the sum of values that have been set as
constants just a few lines above and nothing has changed them. Therefore
remove this.
Reviewed-by: NRich Salz <rsalz@openssl.org>

There were a few remaining references to SSLv2 support which are no longer
relevant now that it has been removed.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

There was a discrepancy between what ciphersuites we allowed to send a
CertificateRequest, and what ciphersuites we allowed to receive one. So
add PSK and SRP to the disallowed ones.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Some functions were marked as inline in statem_srvr.c where they probably
didn't need to be, so remove it.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

|tls_process_finished| was checking that |peer_finish_md_len| was
non-negative. However neither |tls1_final_finish_mac| or
|ssl3_final_finish_mac| can ever return a negative value, so the check is
superfluous.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Due the rest of the state machine changes it makes sense to change the
SSL_state_string return strings from 3* to T*. They are not SSL3 specific
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

There was a few uses of snprintf in the DTLS SCTP code which made more
sense to be a memcpy.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Add the ossl_statem prefix to various funtions to avoid name clashes.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Various enums were introduced as part of the state machine rewrite. As a
matter of style it is preferred for these to be typedefs.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The function dtls1_link_min_mtu() was only used within d1_lib.c so make
it static.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Clang with --strict-warnings was complaining about an uninitalised
variable. In reality it will never be used uninitialised but clang can't
figure out the logic, so just init it anyway to silence the warning.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Rebasing the state machine code introduced a problem with empty
NewSessionTicket processing. The return value from the
tls_process_new_session_ticket() is supposed to be an enum, but a bare
integer was being used. Unfortunately this is valid C so the compiler
doesn't pick it up.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Fix another instance of |al| being unitialised in certain error scenarios.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

A number of error codes were wrong due to a rebase of the state machine
code.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The SSL variable |in_handshake| seems misplaced. It would be better to have
it in the STATEM structure.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

tls_process_client_hello() failed to initialise the |al| variable in some
(error) scenarios. This could cause issues with creating the alert.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Adding the new state machine broke the DTLSv1_listen code because
calling SSL_in_before() was erroneously returning true after DTLSv1_listen
had successfully completed. This change ensures that SSL_in_before returns
false.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Remove repeated blocks of checking SSL and then SSL_CTX for the
info_callback.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Add an OSSL_ prefix onto STATEM
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

SSL_state has been replaced by SSL_get_state and SSL_set_state is no longer
supported.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The |no_cert_verify| should be in the state machine structure not in SSL
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Change various state machine functions to use the prefix ossl_statem
instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Rename the enum HANDSHAKE_STATE to OSSL_HANDSHAKE_STATE to ensure there are
no namespace clashes, and convert it into a typedef.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Fixed some issues in the logic for determining whether an SKE should be
expected or not. In particular only allow an SKE for RSA if its export and
the key size is not allowed. Also fix the ephemeral ciphersuite checks and
add in a missing call to ssl3_check_cert_and_algorithm().
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Fix an out of date reference to old state machine code in a comment
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The next_state variable is no longer needed in the new state machine.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Add some documentation on the thinking behind the state machine.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>