Fixes #4716
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6173)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6229)

Don't call the decrypt ticket callback if we've already encountered a
fatal error. Do call it if we have an empty ticket present.

Change the return code to have 5 distinct returns codes and separate it
from the input status value.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6198)

The recent change in behaviour where you do not get a NewSessionTicket
message sent if you established the connection using a PSK caused a mem
leak to be triggered in sslapitest. It was actually a latent bug and we
were just lucky we never hit it before. The problem is due to complexity
with the way PSK sessions were set up in the early_data tests. PSK session
reference counting was handled differently to normal session reference
counting. This meant there were lots of special cases in the code where
we don't free a session if it is a PSK. It makes things easier if we just
handle PSK reference counts in the same way as other session reference
counts, and then we can remove all of the special case code.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6198)

Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6198)

Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6198)

The return value from the ticket_key callback was not properly handled in
TLSv1.3, so that a ticket was *always* renewed even if the callback
requested that it should not be.

Also the ticket decrypt callback was not being called at all in TLSv1.3.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6198)

When a server call SSL_write_early_data() to write to an unauthenticated
client the buffering BIO is still in place, so we should ensure we flush
the write.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6198)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6159)

This was preventing DTLS connections from being made from the command line.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6159)

The TLS code marks records as read when its finished using a record. The DTLS code did
not do that. However SSL_has_pending() relies on it. So we should make DTLS consistent.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6159)

If using an old style PSK callback and no certificate is configured for
the server, we should prefer ciphersuites based on SHA-256, because that
is the default hash for those callbacks as specified in the TLSv1.3 spec.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6215)

The s_client/s_server docs were missing documentation for this option.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6215)

If we have no certificate and we are using "old style" PSKs then we will
always default to using SHA-256 for that PSK. However we may have selected
a ciphersuite that is not based on SHA-256. Therefore if we see that there
are no certificates and we have been configured for "old style" PSKs then
we should prefer SHA-256 based ciphersuites during the selection process.

Fixes #6197
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6215)

Reviewed-by: NRich Salz <rsalz@openssl.org>
GH: #6208

Reviewed-by: NRich Salz <rsalz@openssl.org>
GH: #6208

In certain circumstances in the DTLS code we have to drop a record (e.g. if
it is a stale retransmit). We then have to move on to try and read the next
record. Some applications using blocking sockets (e.g. s_server/s_client
will hang if there isn't actually any data to be read from the socket yet).
Others can tolerate this. Therefore SSL_read()/SSL_write() can sometimes
return SSL_ERROR_WANT_READ/SSL_ERROR_WANT_WRITE even when using blocking
sockets. Applications can use the mode SSL_MODE_AUTO_RETRY, to switch this
behaviour off so that we never return unless we have read the data we
wanted to.

Commit ad962252 fixed a DTLS problem where we always retried even if
SSL_MODE_AUTO_RETRY was not set. However that fix caused the Boring
ossl_shim to fail in some tests because it was relying on the previous
(buggy) behaviour. The ossl_shim should be set into SSL_MODE_AUTO_RETRY if
it is not operating asynchronously to avoid this problem.

[extended tests]
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6216)

Reviewed-by: NRich Salz <rsalz@openssl.org>

One is clang --strict-warnings and one gcc sanitizer extended test.
Sanitizer build is quite expensive, can take >30 mins and is commented
for occasions when there is reason to believe that PPC-specific problem
can be diagnosed with sanitizer.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6192)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6205)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6205)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6207)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6070)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6070)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6070)

* EC_POINT_mul is now responsible for constant time point multiplication
  (for single fixed or variable point multiplication, when the scalar is
  in the range [0,group_order), so we need to strip the nonce padding
  from ECDSA.
* Entry added to CHANGES
* Updated EC_POINT_mul documentation
  - Integrate existing EC_POINT_mul and EC_POINTs_mul entries in the
    manpage to reflect the shift in constant-time expectations when
    performing a single fixed or variable point multiplication;
  - Add documentation to ec_method_st to reflect the updated "contract"
    between callers and implementations of ec_method_st.mul.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6070)

items->ile3$w_code was checked before it was assigned its value...
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6200)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6191)

Triggered by Coverity analysis.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6190)

Drop a record from a handshake and check that we can still complete the
handshake. Repeat for all records in the handshake.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6170)

During a full handshake the server is the last one to "speak". The timer
should continue to run until we know that the client has received our last
flight (e.g. because we receive some application data).
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6170)

Otherwise we may end up in a hang when using blocking sockets
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6170)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6170)

Fixes #4929
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6170)

Previous tests only invoked CMS via the command line app. This test uses
the CMS API directly to do and encrypt and decrypt operation. This test
would have caught the memory leak fixed by the previous commit (when
building with enable-crypto-mdebug).
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6142)

The function CMS_RecipientInfo_set0_pkey() is a "set0" and therefore
memory management passes to OpenSSL. If the same function is called again
then we should ensure that any previous value that was set is freed first
before we set it again.

Fixes #5052
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6142)

deletion of *.exp files in krb5 sub-module.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6186)

Linking fails with "unrecognized option '--push-state--no-as-needed'",
which is beyond our control.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6185)

Apparently trusty image has newer clang, there is no need to pull
clang-3.9 packages. It's clang-5.0.0, installation is a bit quirky,
as it fails to compile for example strcmp(s,"-") without warning,
and complains about unreferred -I flags. But it's argued that benefits
of exercising newer sanitizer outweights the inconvenience of
additional -D__NO_STRING_INLINE and -Wno-unused-command-line-argument.

Also pull golang when actually needed.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6185)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6183)