Change handshake hash array into a single digest context simplifying the
handhake hash code. Use EVP_md5_sha1() if needed for handshake hashes in
TLS 1.1 and earlier.

Simplify PRF code to also use a single digest and treat EVP_md5_sha1()
as a special case.

Modify algorithm2 field of ciphers to use a single index value for handshake
hash and PRF instead of a bitmap.
Reviewed-by: NMatt Caswell <matt@openssl.org>

Various updates following feedback from the recent commit of the new
GOST2012 code.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>

This patch contains the necessary changes to provide GOST 2012
ciphersuites in TLS. It requires the use of an external GOST 2012 engine.
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>

Make it clear that this function is ssl specific.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Tidy up the libssl async calls and make sure all IO functions are covered.
Reviewed-by: NRich Salz <rsalz@openssl.org>

0 is a valid file descriptor so SSL_get_async_wait_fd should instead return
-1 on error.
Reviewed-by: NRich Salz <rsalz@openssl.org>

The ASYNC_in_job() function is redundant. The same effect can be achieved by
using ASYNC_get_current_job().
Reviewed-by: NRich Salz <rsalz@openssl.org>

Initial API implemented for notifying applications that an ASYNC_JOB
has completed. Currently only s_server is using this. The Dummy Async
engine "cheats" in that it notifies that it has completed *before* it
pauses the job. A normal async engine would not do that.

Only the posix version of this has been implemented so far, so it will
probably fail to compile on Windows at the moment.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Removed the function ASYNC_job_is_waiting() as it was redundant. The only
time user code has a handle on a job is when one is waiting, so all they
need to do is check whether the job is NULL. Also did some cleanups to
make sure the job really is NULL after it has been freed!
Reviewed-by: NRich Salz <rsalz@openssl.org>

The following entry points have been made async aware:
SSL_accept
SSL_read
SSL_write

Also added is a new mode - SSL_MODE_ASYNC. Calling the above functions with
the async mode enabled will initiate a new async job. If an async pause is
encountered whilst executing the job (such as for example if using SHA1/RSA
with the Dummy Async engine), then the above functions return with
SSL_WANT_ASYNC. Calling the functions again (with exactly the same args
as per non-blocking IO), will resume the job where it left off.
Reviewed-by: NRich Salz <rsalz@openssl.org>

There are lots of calls to EVP functions from within libssl There were
various places where we should probably check the return value but don't.
This adds these checks.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The new function SSL_use_certificate_chain_file was always crashing in
the internal function use_certificate_chain_file because it would pass a
NULL value for SSL_CTX *, but use_certificate_chain_file would
unconditionally try to dereference it.
Reviewed-by: NStephen Henson <steve@openssl.org>

if we have a malloc |x = OPENSSL_malloc(...)| sometimes we check |x|
for NULL and sometimes we treat it as a boolean |if(!x) ...|. Standardise
the approach in libssl.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

The SSL object was being deref'd and then there was a later redundant check
to see if it is NULL. We assume all SSL_foo functions pass a non NULL SSL
object and do not check it.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>

There were a few remaining references to SSLv2 support which are no longer
relevant now that it has been removed.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The SSL variable |in_handshake| seems misplaced. It would be better to have
it in the STATEM structure.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The |no_cert_verify| should be in the state machine structure not in SSL
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Change various state machine functions to use the prefix ossl_statem
instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The SSL structure contained a "type" variable that was set to either
SSL_ST_ACCEPT or SSL_ST_CONNECT depending on whether we are the server or
the client. This duplicates the capability of the "server" variable and was
actually rarely used.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Clean up and remove lots of code that is now no longer needed due to the
move to the new state machine.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Split the TLS server ssl3_get_* and ssl3_send_* functions into two ready
for the migration to the new state machine code.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

This is the first drop of the new state machine code.

The rewrite has the following objectives:
- Remove duplication of state code between client and server
- Remove duplication of state code between TLS and DTLS
- Simplify transitions and bring the logic together in a single location
  so that it is easier to validate
- Remove duplication of code between each of the message handling functions
- Receive a message first and then work out whether that is a valid
  transition - not the other way around (the other way causes lots of issues
  where we are expecting one type of message next but actually get something
  else)
- Separate message flow state from handshake state (in order to better
  understand each)
  - message flow state = when to flush buffers; handling restarts in the
    event of NBIO events; handling the common flow of steps for reading a
    message and the common flow of steps for writing a message etc
  - handshake state = what handshake message are we working on now
- Control complexity: only the state machine can change state: keep all
  the state changes local to a file

This builds on previous state machine related work:
- Surface CCS processing in the state machine
- Version negotiation rewrite
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Much related/similar work also done by
Ivan Nestlerode <ivan.nestlerode@sonos.com>

   +Replace FILE BIO's with dummy ops that fail.
   +Include <stdio.h> for sscanf() even with no-stdio (since the declaration
    is there). We rely on sscanf() to parse the OPENSSL_ia32cap environment
    variable, since it can be larger than a 'long'. And we don't rely on the
    availability of strtoull().
   +Remove OPENSSL_stderr(); not used.
   +Make OPENSSL_showfatal() do nothing (currently without stdio there's
    nothing we can do).
   +Remove file-based functionality from ssl/. The function
    prototypes were already gone, but not the functions themselves.
   +Remove unviable conf functionality via SYS_UEFI
   +Add fallback definition of BUFSIZ.
   +Remove functions taking FILE * from header files.
   +Add missing DECLARE_PEM_write_fp_const
   +Disable X509_LOOKUP_hash_dir(). X509_LOOKUP_file() was already compiled out,
    so remove its prototype.
   +Use OPENSSL_showfatal() in CRYPTO_destroy_dynlockid().
   +Eliminate SRP_VBASE_init() and supporting functions. Users will need to
    build the verifier manually instead.
   +Eliminate compiler warning for unused do_pk8pkey_fp().
   +Disable TEST_ENG_OPENSSL_PKEY.
   +Disable GOST engine as is uses [f]printf all over the place.
   +Eliminate compiler warning for unused send_fp_chars().
Signed-off-by: NRich Salz <rsalz@akamai.com>
Reviewed-by: NTim Hudson <tjh@openssl.org>

Previously you could only set both the default path and file locations
together. This adds the ability to set one without the other.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Since SSLv3, a CipherSuite is always 2 bytes. The only place where we
need 3-byte ciphers is SSLv2-compatible ClientHello processing.

So, remove the ssl_put_cipher_by_char indirection.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Make sure it's valid very early.
Signed-off-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

For server use a PSK identity hint value in the CERT structure which
is inherited when SSL_new is called and which allows applications to
set hints on a per-SSL basis. The previous version of
SSL_use_psk_identity_hint tried (wrongly) to use the SSL_SESSION structure.

PR#4039
Reviewed-by: NMatt Caswell <matt@openssl.org>

-Wincompatible-pointer-types-discards-qualifiers (the latter did not require
any code changes).
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

After openssl_zalloc, cleanup more "set to 0/NULL" assignments.
Many are from github feedback.
Reviewed-by: NTim Hudson <tjh@openssl.org>

There are many places (nearly 50) where we malloc and then memset.
Add an OPENSSL_zalloc routine to encapsulate that.
(Missed one conversion; thanks Richard)
Also fixes GH328
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

This was obsolete in 2001.  This is not the same as Gost94 digest.
Thanks to Dmitry Belyavsky <beldmit@gmail.com> for review and advice.
Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

While closing RT3588 (Remove obsolete comment) Kurt and I saw that a
few lines to completely clear the SSL cipher state could be moved into
a common function.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

The new accessors SSL_get_client_random, SSL_get_server_random and
SSL_SESSION_get_master_key should return a size_t to match the type of the
|outlen| parameter.
Reviewed-by: NRichard Levitte <levitte@openssl.org>