if we have a malloc |x = OPENSSL_malloc(...)| sometimes we check |x|
for NULL and sometimes we treat it as a boolean |if(!x) ...|. Standardise
the approach in libssl.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

The SSL object was being deref'd and then there was a later redundant check
to see if it is NULL. We assume all SSL_foo functions pass a non NULL SSL
object and do not check it.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>

There were a few remaining references to SSLv2 support which are no longer
relevant now that it has been removed.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The SSL variable |in_handshake| seems misplaced. It would be better to have
it in the STATEM structure.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The |no_cert_verify| should be in the state machine structure not in SSL
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Change various state machine functions to use the prefix ossl_statem
instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The SSL structure contained a "type" variable that was set to either
SSL_ST_ACCEPT or SSL_ST_CONNECT depending on whether we are the server or
the client. This duplicates the capability of the "server" variable and was
actually rarely used.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Clean up and remove lots of code that is now no longer needed due to the
move to the new state machine.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Split the TLS server ssl3_get_* and ssl3_send_* functions into two ready
for the migration to the new state machine code.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

This is the first drop of the new state machine code.

The rewrite has the following objectives:
- Remove duplication of state code between client and server
- Remove duplication of state code between TLS and DTLS
- Simplify transitions and bring the logic together in a single location
  so that it is easier to validate
- Remove duplication of code between each of the message handling functions
- Receive a message first and then work out whether that is a valid
  transition - not the other way around (the other way causes lots of issues
  where we are expecting one type of message next but actually get something
  else)
- Separate message flow state from handshake state (in order to better
  understand each)
  - message flow state = when to flush buffers; handling restarts in the
    event of NBIO events; handling the common flow of steps for reading a
    message and the common flow of steps for writing a message etc
  - handshake state = what handshake message are we working on now
- Control complexity: only the state machine can change state: keep all
  the state changes local to a file

This builds on previous state machine related work:
- Surface CCS processing in the state machine
- Version negotiation rewrite
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Much related/similar work also done by
Ivan Nestlerode <ivan.nestlerode@sonos.com>

   +Replace FILE BIO's with dummy ops that fail.
   +Include <stdio.h> for sscanf() even with no-stdio (since the declaration
    is there). We rely on sscanf() to parse the OPENSSL_ia32cap environment
    variable, since it can be larger than a 'long'. And we don't rely on the
    availability of strtoull().
   +Remove OPENSSL_stderr(); not used.
   +Make OPENSSL_showfatal() do nothing (currently without stdio there's
    nothing we can do).
   +Remove file-based functionality from ssl/. The function
    prototypes were already gone, but not the functions themselves.
   +Remove unviable conf functionality via SYS_UEFI
   +Add fallback definition of BUFSIZ.
   +Remove functions taking FILE * from header files.
   +Add missing DECLARE_PEM_write_fp_const
   +Disable X509_LOOKUP_hash_dir(). X509_LOOKUP_file() was already compiled out,
    so remove its prototype.
   +Use OPENSSL_showfatal() in CRYPTO_destroy_dynlockid().
   +Eliminate SRP_VBASE_init() and supporting functions. Users will need to
    build the verifier manually instead.
   +Eliminate compiler warning for unused do_pk8pkey_fp().
   +Disable TEST_ENG_OPENSSL_PKEY.
   +Disable GOST engine as is uses [f]printf all over the place.
   +Eliminate compiler warning for unused send_fp_chars().
Signed-off-by: NRich Salz <rsalz@akamai.com>
Reviewed-by: NTim Hudson <tjh@openssl.org>

Previously you could only set both the default path and file locations
together. This adds the ability to set one without the other.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Since SSLv3, a CipherSuite is always 2 bytes. The only place where we
need 3-byte ciphers is SSLv2-compatible ClientHello processing.

So, remove the ssl_put_cipher_by_char indirection.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Make sure it's valid very early.
Signed-off-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

For server use a PSK identity hint value in the CERT structure which
is inherited when SSL_new is called and which allows applications to
set hints on a per-SSL basis. The previous version of
SSL_use_psk_identity_hint tried (wrongly) to use the SSL_SESSION structure.

PR#4039
Reviewed-by: NMatt Caswell <matt@openssl.org>

-Wincompatible-pointer-types-discards-qualifiers (the latter did not require
any code changes).
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

After openssl_zalloc, cleanup more "set to 0/NULL" assignments.
Many are from github feedback.
Reviewed-by: NTim Hudson <tjh@openssl.org>

There are many places (nearly 50) where we malloc and then memset.
Add an OPENSSL_zalloc routine to encapsulate that.
(Missed one conversion; thanks Richard)
Also fixes GH328
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

This was obsolete in 2001.  This is not the same as Gost94 digest.
Thanks to Dmitry Belyavsky <beldmit@gmail.com> for review and advice.
Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

While closing RT3588 (Remove obsolete comment) Kurt and I saw that a
few lines to completely clear the SSL cipher state could be moved into
a common function.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

The new accessors SSL_get_client_random, SSL_get_server_random and
SSL_SESSION_get_master_key should return a size_t to match the type of the
|outlen| parameter.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Change the new SSL_get_client_random(), SSL_get_server_random() and
SSL_SESSION_get_master_key() functions to use size_t for |outlen| instead of
int.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Tor uses these values to implement a low-rent clone of RFC 5705 (which,
in our defense, we came up with before RFC 5705 existed).  But now that
ssl_st is opaque, we need another way to get at them.

Includes documentation, with suitable warnings about not actually
using these functions.
Signed-off-by: NNick Mathewson <nickm@torproject.org>
Signed-off-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NTim Hudson <tjh@openssl.org>

On the server side, if you want to know which ciphers the client
offered, you had to use session->ciphers.  But that field is no
longer visible, so we need a method to get at it.
Signed-off-by: NNick Mathewson <nickm@torproject.org>
Signed-off-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NTim Hudson <tjh@openssl.org>

The certificate masks are used to select which ciphersuite we are going to
use. The variables |emask_k| and |emask_a| relate to export grade key
exchange and authentication respecitively. The variables |mask_k| and
|mask_a| are the equivalent versions for non-export grade. This fixes an
instance where the two usages of export/non-export were mixed up. In
practice it makes little difference since it still works!
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Given the pervasive nature of TLS extensions it is inadvisable to run
OpenSSL without support for them. It also means that maintaining
the OPENSSL_NO_TLSEXT option within the code is very invasive (and probably
not well tested). Therefore it is being removed.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Move per-connection state out of the CERT structure: which should just be
for shared configuration data (e.g. certificates to use).

In particular move temporary premaster secret, raw ciphers, peer signature
algorithms and shared signature algorithms.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Miscellaneous updates following review comments on the version negotiation
rewrite patches.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

This commit changes the way that we do server side protocol version
negotiation. Previously we had a whole set of code that had an "up front"
state machine dedicated to the negotiating the protocol version. This adds
significant complexity to the state machine. Historically the justification
for doing this was the support of SSLv2 which works quite differently to
SSLv3+. However, we have now removed support for SSLv2 so there is little
reason to maintain this complexity.

The one slight difficulty is that, although we no longer support SSLv2, we
do still support an SSLv3+ ClientHello in an SSLv2 backward compatible
ClientHello format. This is generally only used by legacy clients. This
commit adds support within the SSLv3 code for these legacy format
ClientHellos.

Server side version negotiation now works in much the same was as DTLS,
i.e. we introduce the concept of TLS_ANY_VERSION. If s->version is set to
that then when a ClientHello is received it will work out the most
appropriate version to respond with. Also, SSLv23_method and
SSLv23_server_method have been replaced with TLS_method and
TLS_server_method respectively. The old SSLv23* names still exist as
macros pointing at the new name, although they are deprecated.

Subsequent commits will look at client side version negotiation, as well of
removal of the old s23* code.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

Remove RFC2712 Kerberos support from libssl. This code and the associated
standard is no longer considered fit-for-purpose.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Since COMP_METHOD is now defined in comp_lcl.h, it is no
longer possible to create new TLS compression methods without
using the OpenSSL source.  Only ZLIB is supported by default.
Also, since the types are opaque, #ifdef guards to use "char *"
instead of the real type aren't necessary.

The changes are actually minor.  Adding missing copyright to some
files makes the diff misleadingly big.
Reviewed-by: NMatt Caswell <matt@openssl.org>