Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5412)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5663)

With the current mechanism, old cipher strings that used to work in 1.1.0,
may inadvertently disable all TLSv1.3 ciphersuites causing connections to
fail. This is confusing for users.

In reality TLSv1.3 are quite different to older ciphers. They are much
simpler and there are only a small number of them so, arguably, they don't
need the same level of control that the older ciphers have.

This change splits the configuration of TLSv1.3 ciphers from older ones.
By default the TLSv1.3 ciphers are on, so you cannot inadvertently disable
them through your existing config.

Fixes #5359
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5392)

Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/5604)

When the proxy re-encrypted a TLSv1.3 record it was adding a spurious
byte onto the end. This commit removes that.

The "extra" byte was intended to be the inner content type of the record.
However, TLSProxy was actually adding the original encrypted data into the
record (which already has the inner content type in it) and then adding
the spurious additional content type byte on the end (and adjusting the
record length accordingly).

It is interesting to look at why this didn't cause a failure:

The receiving peer first attempts to decrypt the data. Because this is
TLSProxy we always use a GCM based ciphersuite with a 16 byte tag. When
we decrypt this it actually gets diverted to the ossltest engine. All this
does is go through the motions of encrypting/decrypting but just passes
back the original data. Crucially it will never fail because of a bad tag!
The receiving party thinks the spurious additional byte is part of the
tag and the ossltest engine ignores it.

This means the data that gets passed back to the record layer still has
an additional spurious byte on it - but because the 16 byte tag has been
removed, this is actually the first byte of the original tag. Again
because we are using ossltest engine we aren't actually creating "real"
tags - we only ever emit 16, 0 bytes for the tag. So the spurious
additional byte always has the value 0. The TLSv1.3 spec says that records
can have additional 0 bytes on the end of them - this is "padding". So the
record layer interprets this 0 byte as padding and strips it off to end up
with the originally transmitted record data - which it can now process
successfully.
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5370)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5396)

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Add SSL_verify_client_post_handshake() for servers to initiate PHA

Add SSL_force_post_handshake_auth() for clients that don't have certificates
initially configured, but use a certificate callback.

Update SSL_CTX_set_verify()/SSL_set_verify() mode:

* Add SSL_VERIFY_POST_HANDSHAKE to postpone client authentication until after
the initial handshake.

* Update SSL_VERIFY_CLIENT_ONCE now only sends out one CertRequest regardless
of when the certificate authentication takes place; either initial handshake,
re-negotiation, or post-handshake authentication.

Add 'RequestPostHandshake' and 'RequirePostHandshake' SSL_CONF options that
add the SSL_VERIFY_POST_HANDSHAKE to the 'Request' and 'Require' options

Add support to s_client:
* Enabled automatically when cert is configured
* Can be forced enabled via -force_pha

Add support to s_server:
* Use 'c' to invoke PHA in s_server
* Remove some dead code

Update documentation

Update unit tests:
* Illegal use of PHA extension
* TLSv1.3 certificate tests

DTLS and TLS behave ever-so-slightly differently. So, when DTLS1.3 is
implemented, it's PHA support state machine may need to be different.
Add a TODO and a #error

Update handshake context to deal with PHA.

The handshake context for TLSv1.3 post-handshake auth is up through the
ClientFinish message, plus the CertificateRequest message. Subsequent
Certificate, CertificateVerify, and Finish messages are based on this
handshake context (not the Certificate message per se, but it's included
after the hash). KeyUpdate, NewSessionTicket, and prior Certificate
Request messages are not included in post-handshake authentication.

After the ClientFinished message is processed, save off the digest state
for future post-handshake authentication. When post-handshake auth occurs,
copy over the saved handshake context into the "main" handshake digest.
This effectively discards the any KeyUpdate or NewSessionTicket messages
and any prior post-handshake authentication.

This, of course, assumes that the ID-22 did not mean to include any
previous post-handshake authentication into the new handshake transcript.
This is implied by section 4.4.1 that lists messages only up to the
first ClientFinished.
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4964)

The reason to do this is that some output might start with an 'ok',
which TAP catches and takes for TAP output.  The TAP compatible way is
to make all output it shouldn't catch look like comments.

We do this by setting the environment variable HARNESS_OSSL_PREFIX
during tests.  When that is set, apps/openssl uses BIO_f_linebuffer
and sets its prefix to the content of that environment variable.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5224)

We don't need to send this extension in normal operation since
we are our own X.509 library, but add some test cases that force
the extension to be sent and exercise our code to process the extension.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5068)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5068)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5068)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/5110)

On Windows, we sometimes see a behavior with SO_REUSEADDR where there
remains lingering listening sockets on the same address and port as a
newly created one.

To avoid this scenario, we don't create a new proxy port for each new
client run.  Instead, we create one proxy socket when the proxy object
is created, and close it when destroying that object.
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5095)

On Windows, we sometimes see a behavior with SO_REUSEADDR where there
remains lingering listening sockets on the same address and port as a
newly created one.

An easy solution is not to use ReuseAddr on Windows.

Thanks Bernd Edlinger for the suggestion.
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5103)

On Windows, it seems that doing so in a forked (pseudo-)process
sometimes affects the parent, and thereby hides all the results that
are supposed to be seen by the running test framework (the "ok" and
"not ok" lines).

It turns out that our redirection isn't necessary, as the test
framework seems to swallow it all in non-verbose mode anyway.
It's possible that we did need this at some point, but the framework
has undergone some refinement since then...
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5100)

We use the first we can of the following IO::Socket modules to create
sockets:

- IO::Socket::INET6
- IO::Socket::IP
- IO::Socket::INET

The last of them doesn't support IPv6, so if that's the one available,
we must force the s_client and s_server processes to use IPv4.
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/5092)

I received this error from a TLSProxy test:

Failed to start up server (localhost,4443): Transport endpoint is not
connected

So, extend the timeout before we give up trying to connect to the server.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5088)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5071)

Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4701)

Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4701)

Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4701)

Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4701)

The new ServerHello format is essentially now the same as the old TLSv1.2
one, but it must additionally include supported_versions. The version
field is fixed at TLSv1.2, and the version negotiation happens solely via
supported_versions.
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4701)

Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/4701)

Around 138 distinct errors found and fixed; thanks!
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3459)

Normally TLSProxy waits for the s_server process to finish before
continuing. However in cases where serverconnects > 1 we need to keep the
s_server process around for a later test so we continue immediately. This
means that TAP test output can end up being printed to stdout at the same
time as s_server is printing stuff. This confuses the test runner and can
cause spurious test failures. This commit introduces a small delay in cases
where serverconnects > 1 in order to give s_server enough time to finish
what it was doing before we continue to the next test.

Fixes #4129
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4660)

Based on patch from Tomasz Moń:
https://groups.google.com/forum/#!topic/mailing.openssl.dev/fQxXvCg1uQYReviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/1008)

Names were not removed.
Some comments were updated.
Replace Andy's address with openssl.org
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4516)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4507)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4319)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4305)

The program will fail to run if it doesn't exist anyway, no need to
check its existence here.

Fixes #4306
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4309)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4069)

Instead of having perl modules under test/testlib, util and util/perl,
consolidate them all to be inside util/perl.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4069)

Move manpages to manX directories
Add Windows/VMS install fix from Richard Levitte
Update README
Fix typo's
Remove some duplicates
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Adapt util/process_docs.pl
Reviewed-by: NRich Salz <rsalz@openssl.org>