Update the documentation for the newly renamed and modified SSL_set0_rbio()
and SSL_set0_wbio() functions. State that they should be preferred over
SSL_set_bio(). Attempt to document the ownership rules for SSL_set_bio().
Reviewed-by: NRich Salz <rsalz@openssl.org>

SSL_set_rbio() and SSL_set_wbio() are new functions in 1.1.0 and really
should be called SSL_set0_rbio() and SSL_set0_wbio(). The old
implementation was not consistent with what "set0" means though as there
were special cases around what happens if the rbio and wbio are the same.
We were only ever taking one reference on the BIO, and checking everywhere
whether the rbio and wbio are the same so as not to double free.

A better approach is to rename the functions to SSL_set0_rbio() and
SSL_set0_wbio(). If an existing BIO is present it is *always* freed
regardless of whether the rbio and wbio are the same or not. It is
therefore the callers responsibility to ensure that a reference is taken
for *each* usage, i.e. one for the rbio and one for the wbio.

The legacy function SSL_set_bio() takes both the rbio and wbio in one go
and sets them both. We can wrap up the old behaviour in the implementation
of that function, i.e. previously if the rbio and wbio are the same in the
call to this function then the caller only needed to ensure one reference
was passed. This behaviour is retained by internally upping the ref count.

This commit was inspired by BoringSSL commit f715c423224.

RT#4572
Reviewed-by: NRich Salz <rsalz@openssl.org>

This adds some simple SSL BIO tests that check for pushing and popping of
BIOs into the chain. These tests would have caught the bugs fixed in the
previous three commits, if combined with a crypto-mdebug build.
Reviewed-by: NRich Salz <rsalz@openssl.org>

The BIO_pop implementation assumes that the rbio still equals the next BIO
in the chain. While this would normally be the case, it is possible that it
could have been changed directly by the application. It also does not
properly cater for the scenario where the buffering BIO is still in place
for the write BIO.

Most of the existing BIO_pop code for SSL BIOs can be replaced by a single
call to SSL_set_bio(). This is equivalent to the existing code but
additionally handles the scenario where the rbio has been changed or the
buffering BIO is still in place.
Reviewed-by: NRich Salz <rsalz@openssl.org>

When pushing a BIO onto an SSL BIO we set the rbio and wbio for the SSL
object to be the BIO that has been pushed. Therefore we need to up the ref
count for that BIO. The existing code was uping the ref count on the wrong
BIO.
Reviewed-by: NRich Salz <rsalz@openssl.org>

When setting the read bio we free up any old existing one. However this can
lead to a double free if the existing one is the same as the write bio.
Reviewed-by: NRich Salz <rsalz@openssl.org>

The SSL_set_bio() function has some complicated ownership rules. This adds a
test to make sure it all works as expected.
Reviewed-by: NRich Salz <rsalz@openssl.org>

SSLv2 is no longer supported in 1.1.0, however we *do* still accept an SSLv2
style ClientHello, as long as we then subsequently negotiate a protocol
version >= SSLv3. The record format for SSLv2 style ClientHellos is quite
different to SSLv3+. We only accept this format in the first record of an
initial ClientHello. Previously we checked this by confirming
s->first_packet is set and s->server is true. However, this really only
tells us that we are dealing with an initial ClientHello, not that it is
the first record (s->first_packet is badly named...it really means this is
the first message). To check this is the first record of the initial
ClientHello we should also check that we've not received any data yet
(s->init_num == 0), and that we've not had any empty records.

GitHub Issue #1298
Reviewed-by: NEmilia Käsper <emilia@openssl.org>

Cannot nest B<> tags
Document "openssl speed" command.
Fix doc nits: missing NAME/SYNOPSIS stuff
Reviewed-by: NTim Hudson <tjh@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

8605abf1 fixed the nval leak, but it
used free instead of pop_free. nval owns its contents, so it should be
freed with pop_free. See the pop_free call a few lines down.

This is a no-op as, in this codepath, we must have nval == NULL or
sk_CONF_VALUE_num(nval) == 0. In those cases, free and pop_free are
identical. However, variables should be freed consistently.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1351)

Reviewed-by: NRich Salz <rsalz@openssl.org>
GH: #1324

Reviewed-by: NRich Salz <rsalz@openssl.org>
GH: #1345

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Instead of X509_LU_FAIL, X509_LU_RETRY use 0/1 for return values.

RT#4577
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

X509_LU_RETRY and X509_LU_FAIL are not X509_OBJECT types so don't include
them in the enum.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Remove current_method: it was intended as a means of retrying
lookups bit it was never used. Now that X509_verify_cert() is
a "one shot" operation it can never work as intended.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Simplify BIO init using OPENSSL_zalloc().
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1261)

Some versions of the Microsoft TLS stack have problems when the DHE public key
is encoded with fewer bytes than the DHE prime.

There's some public acknowledgement of the bug at these links:

https://connect.microsoft.com/IE/feedback/details/1253526/tls-serverkeyexchange-with-1024-dhe-may-encode-dh-y-as-127-bytes-breaking-internet-explorer-11
https://connect.microsoft.com/IE/feedback/details/1104905/wininet-calculation-of-mac-in-tls-handshake-intermittently-fails-for-dhe-rsa-key-exchange

This encoding issue also causes the same errors with 2048-bit DHE, if the
public key is encoded in fewer than 256 bytes and includes the TLS stack on
Windows Phone 8.x.
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1320)

This adds the functions X509_set_proxy_pathlen(), which sets the
internal pc path length cache for a given X509 structure, along with
X509_get_proxy_pathlen(), which retrieves it.

Along with the previously added X509_set_proxy_flag(), this provides
the tools needed to manipulate all the information cached on proxy
certificates, allowing external code to do what's necessary to have
them verified correctly by the libcrypto code.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Since there are a number of function pointers in X509_STORE that might
lead to user code, it makes sense for them to be able to lock the
store while they do their work.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

We only add setters for X509_STORE function pointers except for the
verify callback function.  The thought is that the function pointers
in X509_STORE_CTX are a cache for the X509_STORE functions.
Therefore, it's preferable if the user makes the changes in X509_STORE
before X509_STORE_CTX_init is called, and otherwise use the verify
callback to override any results from OpenSSL's internal
calculations.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NMatt Caswell <matt@openssl.org>
Signed-off-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Break two long messages.
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Old inactive inherited code, a code relic for sure.
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1301)

Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

X509_REVOKED_get0_extensions
X509_check_private_key
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)

Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1300)