To avoid confusion with the read PACKET structure.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

DTLS can handle out of order record delivery. Additionally since
handshake messages can be bigger than will fit into a single packet, the
messages can be fragmented across multiple records (as with normal TLS).
That means that the messages can arrive mixed up, and we have to
reassemble them. We keep a queue of buffered messages that are "from the
future", i.e. messages we're not ready to deal with yet but have arrived
early. The messages held there may not be full yet - they could be one
or more fragments that are still in the process of being reassembled.

The code assumes that we will eventually complete the reassembly and
when that occurs the complete message is removed from the queue at the
point that we need to use it.

However, DTLS is also tolerant of packet loss. To get around that DTLS
messages can be retransmitted. If we receive a full (non-fragmented)
message from the peer after previously having received a fragment of
that message, then we ignore the message in the queue and just use the
non-fragmented version. At that point the queued message will never get
removed.

Additionally the peer could send "future" messages that we never get to
in order to complete the handshake. Each message has a sequence number
(starting from 0). We will accept a message fragment for the current
message sequence number, or for any sequence up to 10 into the future.
However if the Finished message has a sequence number of 2, anything
greater than that in the queue is just left there.

So, in those two ways we can end up with "orphaned" data in the queue
that will never get removed - except when the connection is closed. At
that point all the queues are flushed.

An attacker could seek to exploit this by filling up the queues with
lots of large messages that are never going to be used in order to
attempt a DoS by memory exhaustion.

I will assume that we are only concerned with servers here. It does not
seem reasonable to be concerned about a memory exhaustion attack on a
client. They are unlikely to process enough connections for this to be
an issue.

A "long" handshake with many messages might be 5 messages long (in the
incoming direction), e.g. ClientHello, Certificate, ClientKeyExchange,
CertificateVerify, Finished. So this would be message sequence numbers 0
to 4. Additionally we can buffer up to 10 messages in the future.
Therefore the maximum number of messages that an attacker could send
that could get orphaned would typically be 15.

The maximum size that a DTLS message is allowed to be is defined by
max_cert_list, which by default is 100k. Therefore the maximum amount of
"orphaned" memory per connection is 1500k.

Message sequence numbers get reset after the Finished message, so
renegotiation will not extend the maximum number of messages that can be
orphaned per connection.

As noted above, the queues do get cleared when the connection is closed.
Therefore in order to mount an effective attack, an attacker would have
to open many simultaneous connections.

Issue reported by Quan Luo.

CVE-2016-2179
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Run util/openssl-format-source on ssl/

Some comments and hand-formatted tables were fixed up
manually by disabling auto-formatting.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

DTLSv1_client_method() is deprecated, but it was the only way to obtain
DTLS1_BAD_VER support. The SSL_OP_CISCO_ANYCONNECT hack doesn't work with
DTLS_client_method(), and it's relatively non-trivial to make it work without
expanding the hack into lots of places.

So deprecate SSL_OP_CISCO_ANYCONNECT with DTLSv1_client_method(), and make
it work with SSL_CTX_set_{min,max}_proto_version(DTLS1_BAD_VER) instead.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

Fix some indentation at the same time
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1292)

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NKurt Roeckx <kurt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1264)

Reviewed-by: NKurt Roeckx <kurt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1168)

ChangeCipherSpec messages have a defined value. They also may not occur
in the middle of a handshake message. The current logic will accept a
ChangeCipherSpec with value 2. It also would accept up to three bytes of
handshake data before the ChangeCipherSpec which it would discard
(because s->init_num gets reset).

Instead, require that s->init_num is 0 when a ChangeCipherSpec comes in.

RT#4391
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

Set ctx->error = X509_V_ERR_OUT_OF_MEM when verificaiton cannot
continue due to malloc failure.  Also, when X509_verify_cert()
returns <= 0 make sure that the verification status does not remain
X509_V_OK, as a last resort set it it to X509_V_ERR_UNSPECIFIED,
just in case some code path returns an error without setting an
appropriate value of ctx->error.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

We now send the highest supported version by the client, even if the session
uses an older version.

This fixes 2 problems:
- When you try to reuse a session but the other side doesn't reuse it and
  uses a different protocol version the connection will fail.
- When you're trying to reuse a session with an old version you might be
  stuck trying to reuse the old version while both sides support a newer
  version
Signed-off-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

GH: #852, MR: #2452

Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

MR: #1824

Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

MR: #1595

Adjust ssl_set_client_hello_version to get both the minimum and maximum and then
make ssl_set_client_hello_version use the maximum version.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

MR: #1595

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

This was done by the following
        find . -name '*.[ch]' | /tmp/pl
where /tmp/pl is the following three-line script:
        print unless $. == 1 && m@/\* .*\.[ch] \*/@;
        close ARGV if eof; # Close file to reset $.

And then some hand-editing of other files.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Some users want to disable SSL 3.0/TLS 1.0/TLS 1.1, and enable just
TLS 1.2.  In the future they might want to disable TLS 1.2 and
enable just TLS 1.3, ...

This commit makes it possible to disable any or all of the TLS or
DTLS protocols.  It also considerably simplifies the SSL/TLS tests,
by auto-generating the min/max version tests based on the set of
supported protocols (425 explicitly written out tests got replaced
by two loops that generate all 425 tests if all protocols are
enabled, fewer otherwise).
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NViktor Dukhovni <openssl-users@dukhovni.org>

The protocol selection code is now consolidated in a few consecutive
short functions in a single file and is table driven.  Protocol-specific
constraints that influence negotiation are moved into the flags
field of the method structure.  The same protocol version constraints
are now applied in all code paths.  It is now much easier to add
new protocol versions without reworking the protocol selection
logic.

In the presence of "holes" in the list of enabled client protocols
we no longer select client protocols below the hole based on a
subset of the constraints and then fail shortly after when it is
found that these don't meet the remaining constraints (suiteb, FIPS,
security level, ...).  Ideally, with the new min/max controls users
will be less likely to create "holes" in the first place.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

Remove all fixed DH ciphersuites and associated logic.
Reviewed-by: NMatt Caswell <matt@openssl.org>

Various updates following feedback from the recent commit of the new
GOST2012 code.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

This patch contains the necessary changes to provide GOST 2012
ciphersuites in TLS. It requires the use of an external GOST 2012 engine.
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>

We were setting |s->renegotiate| and |s->new_session| to 0 twice in
tls_finish_handshake. This is redundant so now we just do it once!
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

We finish the handshake when we move into the TLS_ST_OK state. At various
points we were also unnecessarily finishing it when we were reading/writing
the Finished message. It's much simpler just to do it in TLS_ST_OK, so
remove the other calls.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

|tls_process_finished| was checking that |peer_finish_md_len| was
non-negative. However neither |tls1_final_finish_mac| or
|ssl3_final_finish_mac| can ever return a negative value, so the check is
superfluous.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Various enums were introduced as part of the state machine rewrite. As a
matter of style it is preferred for these to be typedefs.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Change various state machine functions to use the prefix ossl_statem
instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Move some function definitions around within the state machine to make sure
they are in the correct files. Also create a statem_locl.h header for stuff
entirely local to the state machine code and move various definitions into
it.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Pull out the state machine into a separate sub directory. Also moved some
functions which were nothing to do with the state machine but were in state
machine files. Pulled all the SSL_METHOD definitions into one place...most
of those files had very little left in them any more.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Previously each message specific process function would create its own
PACKET structure. Rather than duplicate all of this code lots of times we
should create it in the state machine itself.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The SSL structure contained a "type" variable that was set to either
SSL_ST_ACCEPT or SSL_ST_CONNECT depending on whether we are the server or
the client. This duplicates the capability of the "server" variable and was
actually rarely used.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Clean up and remove lots of code that is now no longer needed due to the
move to the new state machine.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Move all DTLS client side processing into the new state machine code. A
subsequent commit will clean up the old dead code.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

This swaps the implementation of the client TLS state machine to use the
new state machine code instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>