During a DTLS handshake we may get records destined for the next epoch
arrive before we have processed the CCS. In that case we can't decrypt or
verify the record yet, so we buffer it for later use. When we do receive
the CCS we work through the queue of unprocessed records and process them.

Unfortunately the act of processing wipes out any existing packet data
that we were still working through. This includes any records from the new
epoch that were in the same packet as the CCS. We should only process the
buffered records if we've not got any data left.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Run util/openssl-format-source on ssl/

Some comments and hand-formatted tables were fixed up
manually by disabling auto-formatting.
Reviewed-by: NRich Salz <rsalz@openssl.org>

The read pipelining code broke constant time unpadding. See GitHub
issue #1438
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

If we receive a non-initial record but the version number isn't right then
we should send an alert.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Feedback on the previous SSLv2 ClientHello processing fix was that it
breaks layering by reading init_num in the record layer. It also does not
detect if there was a previous non-fatal warning.

This is an alternative approach that directly tracks in the record layer
whether this is the first record.

GitHub Issue #1298
Reviewed-by: NTim Hudson <tjh@openssl.org>

Thanks to Peter Gijsels for pointing out that if a CBC record has 255
bytes of padding, the first was not being checked.

(This is an import of change 80842bdb from BoringSSL.)
Reviewed-by: NEmilia Käsper <emilia@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1431)

SSLv2 is no longer supported in 1.1.0, however we *do* still accept an SSLv2
style ClientHello, as long as we then subsequently negotiate a protocol
version >= SSLv3. The record format for SSLv2 style ClientHellos is quite
different to SSLv3+. We only accept this format in the first record of an
initial ClientHello. Previously we checked this by confirming
s->first_packet is set and s->server is true. However, this really only
tells us that we are dealing with an initial ClientHello, not that it is
the first record (s->first_packet is badly named...it really means this is
the first message). To check this is the first record of the initial
ClientHello we should also check that we've not received any data yet
(s->init_num == 0), and that we've not had any empty records.

GitHub Issue #1298
Reviewed-by: NEmilia Käsper <emilia@openssl.org>

Fix some indentation at the same time
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1292)

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Previously if we received an empty record we just threw it away and
ignored it. Really though if we get an empty record of a different content
type to what we are expecting then that should be an error, i.e. we should
reject out of context empty records. This commit makes the necessary changes
to achieve that.

RT#4395
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

In the SSLV2ClientHello processing code in ssl3_get_record, the value of
|num_recs| will always be 0. This isn't obvious from the code so a comment
is added to explain it.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

The function ssl3_get_record() can obtain multiple records in one go
as long as we are set up for pipelining and all the records are app
data records. The logic in the while loop which reads in each record is
supposed to only continue looping if the last record we read was app data
and we have an app data record waiting in the buffer to be processed. It
was actually checking that the first record had app data and we have an
app data record waiting. This actually amounts to the same thing so wasn't
wrong - but it looks a bit odd because it uses the |rr| array without an
offset.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

Pipelining introduced the concept of multiple records being read in one
go. Therefore we work with an array of SSL3_RECORD objects. The pipelining
change erroneously made a change in ssl3_get_record() to apply the current
record offset to the SSL3_BUFFER we are using for reading. This is wrong -
there is only ever one read buffer. This reverts that change. In practice
this should make little difference because the code block in question is
only ever used when we are processing a single record.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

The numpipes argument to ssl3_enc/tls1_enc is actually the number of
records passed in the array. To make this clearer rename the argument to
|n_recs|.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Rename the have_whole_app_data_record_waiting() function to include the
ssl3_record prefix...and make it a bit shorter.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Update a comment that was out of date due to the pipelining changes
Reviewed-by: NTim Hudson <tjh@openssl.org>

With read pipelining we use multiple SSL3_RECORD structures for reading.
There are SSL_MAX_PIPELINES (32) of them defined (typically not all of these
would be used). Each one has a 16k compression buffer allocated! This
results in a significant amount of memory being consumed which, most of the
time, is not needed.  This change swaps the allocation of the compression
buffer to be lazy so that it is only done immediately before it is actually
used.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Read pipelining is controlled in a slightly different way than with write
pipelining. While reading we are constrained by the number of records that
the peer (and the network) can provide to us in one go. The more records
we can get in one go the more opportunity we have to parallelise the
processing.

There are two parameters that affect this:
* The number of pipelines that we are willing to process in one go. This is
controlled by max_pipelines (as for write pipelining)
* The size of our read buffer. A subsequent commit will provide an API for
adjusting the size of the buffer.

Another requirement for this to work is that "read_ahead" must be set. The
read_ahead parameter will attempt to read as much data into our read buffer
as the network can provide. Without this set, data is read into the read
buffer on demand. Setting the max_pipelines parameter to a value greater
than 1 will automatically also turn read_ahead on.

Finally, the read pipelining as currently implemented will only parallelise
the processing of application data records. This would only make a
difference for renegotiation so is unlikely to have a significant impact.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Use the new pipeline cipher capability to encrypt multiple records being
written out all in one go. Two new SSL/SSL_CTX parameters can be used to
control how this works: max_pipelines and split_send_fragment.

max_pipelines defines the maximum number of pipelines that can ever be used
in one go for a single connection. It must always be less than or equal to
SSL_MAX_PIPELINES (currently defined to be 32). By default only one
pipeline will be used (i.e. normal non-parallel operation).

split_send_fragment defines how data is split up into pipelines. The number
of pipelines used will be determined by the amount of data provided to the
SSL_write call divided by split_send_fragment. For example if
split_send_fragment is set to 2000 and max_pipelines is 4 then:
SSL_write called with 0-2000 bytes == 1 pipeline used
SSL_write called with 2001-4000 bytes == 2 pipelines used
SSL_write called with 4001-6000 bytes == 3 pipelines used
SSL_write_called with 6001+ bytes == 4 pipelines used

split_send_fragment must always be less than or equal to max_send_fragment.
By default it is set to be equal to max_send_fragment. This will mean that
the same number of records will always be created as would have been
created in the non-parallel case, although the data will be apportioned
differently. In the parallel case data will be spread equally between the
pipelines.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Add -DBIO_DEBUG to --strict-warnings.
Remove comments about outdated debugging ifdef guards.
Remove md_rand ifdef guarding an assert; it doesn't seem used.
Remove the conf guards in conf_api since we use OPENSSL_assert, not assert.
For pkcs12 stuff put OPENSSL_ in front of the macro name.
Merge TLS_DEBUG into SSL_DEBUG.
Various things just turned on/off asserts, mainly for checking non-NULL
arguments, which is now removed: camellia, bn_ctx, crypto/modes.
Remove some old debug code, that basically just printed things to stderr:
  DEBUG_PRINT_UNKNOWN_CIPHERSUITES, DEBUG_ZLIB, OPENSSL_RI_DEBUG,
  RL_DEBUG, RSA_DEBUG, SCRYPT_DEBUG.
Remove OPENSSL_SSL_DEBUG_BROKEN_PROTOCOL.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

This was done by the following
        find . -name '*.[ch]' | /tmp/pl
where /tmp/pl is the following three-line script:
        print unless $. == 1 && m@/\* .*\.[ch] \*/@;
        close ARGV if eof; # Close file to reset $.

And then some hand-editing of other files.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Suggested by David Benjamin
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NViktor Dukhovni <openssl-users@dukhovni.org>

MR: #1520

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

There are lots of calls to EVP functions from within libssl There were
various places where we should probably check the return value but don't.
This adds these checks.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

An OPENSSL_assert was being used which could fail (e.g. on a malloc
failure).
Reviewed-by: NRich Salz <rsalz@openssl.org>

The SSL variable |in_handshake| seems misplaced. It would be better to have
it in the STATEM structure.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The old implementation of DTLSv1_listen which has now been replaced still
had a few vestiges scattered throughout the code. This commit removes them.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

This is a workaround so old that nobody remembers what buggy clients
it was for. It's also been broken in stable branches for two years and
nobody noticed (see
https://boringssl-review.googlesource.com/#/c/1694/).
Reviewed-by: NTim Hudson <tjh@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

If the record received is for a version that we don't support, previously we
were sending an alert back. However if the incoming record already looks
like an alert then probably we shouldn't do that. So suppress an outgoing
alert if it looks like we've got one incoming.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

The function RECORD_LAYER_clear() is supposed to clear the contents of the
RECORD_LAYER structure, but retain certain data such as buffers that are
allocated. Unfortunately one buffer (for compression) got missed and was
inadvertently being wiped, thus causing a memory leak.

In part this is due to the fact that RECORD_LAYER_clear() was reaching
inside SSL3_BUFFERs and SSL3_RECORDs, which it really shouldn't. So, I've
rewritten it to only clear the data it knows about, and to defer clearing
of SSL3_RECORD and SSL3_BUFFER structures to SSL_RECORD_clear() and the
new function SSL3_BUFFER_clear().
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>