Some misc return value checks
Reviewed-by: NRich Salz <rsalz@openssl.org>

The ssl3_init_finished_mac() function can fail, in which case we need to
propagate the error up through the stack.

RT#3198
Reviewed-by: NRich Salz <rsalz@openssl.org>

The ssl3_digest_cached_records() function was failing to handle errors
that might be returned from EVP_DigestSignInit() and
EVP_DigestSignUpdate().

RT#4180
Reviewed-by: NStephen Henson <steve@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/952)

This reverts commit 620d540b.
It wasn't reviewed.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>

With read pipelining we use multiple SSL3_RECORD structures for reading.
There are SSL_MAX_PIPELINES (32) of them defined (typically not all of these
would be used). Each one has a 16k compression buffer allocated! This
results in a significant amount of memory being consumed which, most of the
time, is not needed.  This change swaps the allocation of the compression
buffer to be lazy so that it is only done immediately before it is actually
used.
Reviewed-by: NTim Hudson <tjh@openssl.org>

1) Simplify code with better PACKET methods.

2) Make broken SNI parsing explicit. SNI was intended to be extensible
to new name types but RFC 4366 defined the syntax inextensibly, and
OpenSSL has never parsed SNI in a way that would allow adding a new name
type. RFC 6066 fixed the definition but due to broken implementations
being widespread, it appears impossible to ever extend SNI.

3) Annotate resumption behaviour. OpenSSL doesn't currently handle all
extensions correctly upon resumption. Annotate for further clean-up.

4) Send an alert on ALPN protocol mismatch.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

This was done by the following
        find . -name '*.[ch]' | /tmp/pl
where /tmp/pl is the following three-line script:
        print unless $. == 1 && m@/\* .*\.[ch] \*/@;
        close ARGV if eof; # Close file to reset $.

And then some hand-editing of other files.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

MR: #364

Reviewed-by: NMatt Caswell <matt@openssl.org>

Change handshake hash array into a single digest context simplifying the
handhake hash code. Use EVP_md5_sha1() if needed for handshake hashes in
TLS 1.1 and earlier.

Simplify PRF code to also use a single digest and treat EVP_md5_sha1()
as a special case.

Modify algorithm2 field of ciphers to use a single index value for handshake
hash and PRF instead of a bitmap.
Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>

There are lots of calls to EVP functions from within libssl There were
various places where we should probably check the return value but don't.
This adds these checks.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

We always free the handshake buffer when digests are freed so move
it into ssl_free_digest_list()
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

Rewrite ssl3_digest_cached_records handling. Only digest cached records
if digest array is NULL: this means it is safe to call
ssl3_digest_cached_records multiple times (subsequent calls are no op).

Remove flag TLS1_FLAGS_KEEP_HANDSHAKE instead only update handshake buffer
if digest array is NULL.

Add additional "keep" parameter to ssl3_digest_cached_records to indicate
if the handshake buffer should be retained after digesting cached records
(needed for TLS 1.2 client authentication).
Reviewed-by: NMatt Caswell <matt@openssl.org>

This reverts commit d480e182.

Commit broke TLS handshakes due to fragility of digest caching: that will be
fixed separately.
Reviewed-by: NRich Salz <rsalz@openssl.org>

We always free the handshake buffer when digests are freed so move
it into ssl_free_digest_list()
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Where we called openssl_cleanse, make sure we do it on all error
paths.  Be consistent in use of sizeof(foo) when possible.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Just as with the OPENSSL_malloc calls, consistently use sizeof(*ptr)
for memset and memcpy.  Remove needless casts for those functions.
For memset, replace alternative forms of zero with 0.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

For a local variable:
        TYPE *p;
Allocations like this are "risky":
        p = OPENSSL_malloc(sizeof(TYPE));
if the type of p changes, and the malloc call isn't updated, you
could get memory corruption.  Instead do this:
        p = OPENSSL_malloc(sizeof(*p));
Also fixed a few memset() calls that I noticed while doing this.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Don't check for NULL before calling free functions. This gets:
        ERR_STATE_free
        ENGINE_free
        DSO_free
        CMAC_CTX_free
        COMP_CTX_free
        CONF_free
        NCONF_free NCONF_free_data _CONF_free_data
        A sk_free use within OBJ_sigid_free
        TS_TST_INFO_free (rest of TS_ API was okay)
        Doc update for UI_free (all uses were fine)
        X509V3_conf_free
        X509V3_section_free
        X509V3_string_free
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Add OPENSSL_clear_free which merges cleanse and free.
(Names was picked to be similar to BN_clear_free, etc.)
Removed OPENSSL_freeFunc macro.
Fixed the small simple ones that are left:
        CRYPTO_free CRYPTO_free_locked OPENSSL_free_locked
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NMatt Caswell <matt@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

This commit handles BIO_ACCEPT_free BIO_CB_FREE BIO_CONNECT_free
BIO_free BIO_free_all BIO_vfree
Reviewed-by: NMatt Caswell <matt@openssl.org>

Ensure that all functions have their return values checked where
appropriate. This covers all functions defined and called from within
libssl.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Cleanse various intermediate buffers used by the PRF.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>