DTLSv1_client_method() is deprecated, but it was the only way to obtain
DTLS1_BAD_VER support. The SSL_OP_CISCO_ANYCONNECT hack doesn't work with
DTLS_client_method(), and it's relatively non-trivial to make it work without
expanding the hack into lots of places.

So deprecate SSL_OP_CISCO_ANYCONNECT with DTLSv1_client_method(), and make
it work with SSL_CTX_set_{min,max}_proto_version(DTLS1_BAD_VER) instead.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>

Fix some indentation at the same time
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1292)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NKurt Roeckx <kurt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1264)

Reviewed-by: NKurt Roeckx <kurt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1245)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

Based on an orignal commit by GitHub user BertramScharpf. Rebased and
updated to take account of all the updates since this was first raised.

GH PR#62
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/952)

This reverts commit 620d540b.
It wasn't reviewed.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>

Misc fixes for no-sock
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

MR: #1595

Reviewed-by: NRichard Levitte <levitte@openssl.org>

To enable heartbeats for DTLS, configure with enable-heartbeats.
Heartbeats for TLS have been completely removed.

This addresses RT 3647
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Adds a set of tests for the newly rewritten DTLSv1_listen function.
The test pokes various packets at the function and then checks
the return value and the data written out to ensure it is what we
would have expected.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

The recently rewriten DTLSv1_listen code does not support fragmented
ClientHello messages because fragment reassembly requires server state
which is against the whole point of DTLSv1_listen. This change adds some
partial support for fragmented ClientHellos. It requires that the cookie
must be within the initial fragment. That way any non-initial ClientHello
fragments can be dropped and fragment reassembly is not required.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

The DTLSv1_listen function exposed details of the underlying BIO
abstraction and did not properly allow for IPv6. This commit changes the
"peer" argument to be a BIO_ADDR and makes it a first class function
(rather than a ctrl) to ensure proper type checking.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The control commands that previously took a struct sockaddr * have
been changed to take a BIO_ADDR * instead.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

PACKET contents should be read-only. To achieve this, also
- constify two user callbacks
- constify BUF_reverse.
Reviewed-by: NRich Salz <rsalz@openssl.org>

This was done by the following
        find . -name '*.[ch]' | /tmp/pl
where /tmp/pl is the following three-line script:
        print unless $. == 1 && m@/\* .*\.[ch] \*/@;
        close ARGV if eof; # Close file to reset $.

And then some hand-editing of other files.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

This is an internal facility, never documented, not for
public consumption.  Move it into ssl (where it's only used
for DTLS).

I also made the typedef's for pqueue and pitem follow our style: they
name structures, not pointers.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

The protocol selection code is now consolidated in a few consecutive
short functions in a single file and is table driven.  Protocol-specific
constraints that influence negotiation are moved into the flags
field of the method structure.  The same protocol version constraints
are now applied in all code paths.  It is now much easier to add
new protocol versions without reworking the protocol selection
logic.

In the presence of "holes" in the list of enabled client protocols
we no longer select client protocols below the hole based on a
subset of the constraints and then fail shortly after when it is
found that these don't meet the remaining constraints (suiteb, FIPS,
security level, ...).  Ideally, with the new min/max controls users
will be less likely to create "holes" in the first place.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>

if we have a malloc |x = OPENSSL_malloc(...)| sometimes we check |x|
for NULL and sometimes we treat it as a boolean |if(!x) ...|. Standardise
the approach in libssl.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

This OPENSSL_assert in (d)tls1_hearbeat is trivially always going to be
true because it is testing the sum of values that have been set as
constants just a few lines above and nothing has changed them. Therefore
remove this.
Reviewed-by: NRich Salz <rsalz@openssl.org>

The function dtls1_link_min_mtu() was only used within d1_lib.c so make
it static.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The SSL variable |in_handshake| seems misplaced. It would be better to have
it in the STATEM structure.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Adding the new state machine broke the DTLSv1_listen code because
calling SSL_in_before() was erroneously returning true after DTLSv1_listen
had successfully completed. This change ensures that SSL_in_before returns
false.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Change various state machine functions to use the prefix ossl_statem
instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Pull out the state machine into a separate sub directory. Also moved some
functions which were nothing to do with the state machine but were in state
machine files. Pulled all the SSL_METHOD definitions into one place...most
of those files had very little left in them any more.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The DTLSv1_listen code set the state value explicitly to move into init.
Change to use state_set_in_init() instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Note that this commit constifies a user callback parameter and therefore
will break compilation for applications using this callback. But unless
they are abusing write access to the buffer, the fix is trivial.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Add a sanity check that the cookie_len returned by app_gen_cookie_cb is
valid.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

The existing implementation of DTLSv1_listen() is fundamentally flawed. This
function is used in DTLS solutions to listen for new incoming connections
from DTLS clients. A client will send an initial ClientHello. The server
will respond with a HelloVerifyRequest containing a unique cookie. The
client the responds with a second ClientHello - which this time contains the
cookie.

Once the cookie has been verified then DTLSv1_listen() returns to user code,
which is typically expected to continue the handshake with a call to (for
example) SSL_accept().

Whilst listening for incoming ClientHellos, the underlying BIO is usually in
an unconnected state. Therefore ClientHellos can come in from *any* peer.
The arrival of the first ClientHello without the cookie, and the second one
with it, could be interspersed with other intervening messages from
different clients.

The whole purpose of this mechanism is as a defence against DoS attacks. The
idea is to avoid allocating state on the server until the client has
verified that it is capable of receiving messages at the address it claims
to come from. However the existing DTLSv1_listen() implementation completely
fails to do this. It attempts to super-impose itself on the standard state
machine and reuses all of this code. However the standard state machine
expects to operate in a stateful manner with a single client, and this can
cause various problems.

A second more minor issue is that the return codes from this function are
quite confused, with no distinction made between fatal and non-fatal errors.
Most user code treats all errors as non-fatal, and simply retries the call
to DTLSv1_listen().

This commit completely rewrites the implementation of DTLSv1_listen() and
provides a stand alone implementation that does not rely on the existing
state machine. It also provides more consistent return codes.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

There are many places (nearly 50) where we malloc and then memset.
Add an OPENSSL_zalloc routine to encapsulate that.
(Missed one conversion; thanks Richard)
Also fixes GH328
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Especially since after the #ifdef cleanups this is not useful.
Reviewed-by: NMatt Caswell <matt@openssl.org>

Just as with the OPENSSL_malloc calls, consistently use sizeof(*ptr)
for memset and memcpy.  Remove needless casts for those functions.
For memset, replace alternative forms of zero with 0.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

For a local variable:
        TYPE *p;
Allocations like this are "risky":
        p = OPENSSL_malloc(sizeof(TYPE));
if the type of p changes, and the malloc call isn't updated, you
could get memory corruption.  Instead do this:
        p = OPENSSL_malloc(sizeof(*p));
Also fixed a few memset() calls that I noticed while doing this.
Reviewed-by: NRichard Levitte <levitte@openssl.org>