PKCS #1 v2.0 is the name of a document which specifies an algorithm
RSASSA-PKCS1-v1_5, often referred to as "PKCS #1 v1.5" after an earlier
document which specified it. This gets further confusing because the
document PKCS #1 v2.1 specifies two signature algorithms,
RSASSA-PKCS1-v1_5 and RSASSA-PSS. RSA_sign implements RSASSA-PKCS1-v1_5.

Refer to the document using the RFC number which is easier to find
anyway, and refer to the algorithm by its name.
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NRich Salz <rsalz@openssl.org>

GH: #1474

RFC 3447, section 8.2.2, steps 3 and 4 states that verifiers must encode
the DigestInfo struct and then compare the result against the public key
operation result. This implies that one and only one encoding is legal.

OpenSSL instead parses with crypto/asn1, then checks that the encoding
round-trips, and allows some variations for the parameter. Sufficient
laxness in this area can allow signature forgeries, as described in
https://www.imperialviolet.org/2014/09/26/pkcs1.html

Although there aren't known attacks against OpenSSL's current scheme,
this change makes OpenSSL implement the algorithm as specified. This
avoids the uncertainty and, more importantly, helps grow a healthy
ecosystem. Laxness beyond the spec, particularly in implementations
which enjoy wide use, risks harm to the ecosystem for all. A signature
producer which only tests against OpenSSL may not notice bugs and
accidentally become widely deployed. Thus implementations have a
responsibility to honor the specification as tightly as is practical.

In some cases, the damage is permanent and the spec deviation and
security risk becomes a tax all implementors must forever pay, but not
here. Both BoringSSL and Go successfully implemented and deployed
RSASSA-PKCS1-v1_5 as specified since their respective beginnings, so
this change should be compatible enough to pin down in future OpenSSL
releases.

See also https://tools.ietf.org/html/draft-thomson-postel-was-wrong-00

As a bonus, by not having to deal with sign/verify differences, this
version is also somewhat clearer. It also more consistently enforces
digest lengths in the verify_recover codepath. The NID_md5_sha1 codepath
wasn't quite doing this right.
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NRich Salz <rsalz@openssl.org>

GH: #1474

This partially reverts commit c636c1c4. It also tweaks the documentation
and comments in this area. On the client side the documented interface for
SSL_CTX_set_verify()/SSL_set_verify() is that setting the flag
SSL_VERIFY_PEER causes verfication of the server certificate to take place.
Previously what was implemented was that if *any* flag was set then
verification would take place. The above commit improved the semantics to
be as per the documented interface.

However, we have had a report of at least one application where an
application was incorrectly using the interface and used *only*
SSL_VERIFY_FAIL_IF_NO_PEER_CERT on the client side. In OpenSSL prior to
the above commit this still caused verification of the server certificate
to take place. After this commit the application silently failed to verify
the server certificate.

Ideally SSL_CTX_set_verify()/SSL_set_verify() could be modified to indicate
if invalid flags were being used. However these are void functions!

The simplest short term solution is to revert to the previous behaviour
which at least means we "fail closed" rather than "fail open".

Thanks to Cory Benfield for reporting this issue.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

1) Remove some unnecessary fixtures
2) Add EXECUTE_TEST_NO_TEARDOWN shorthand when a fixture exists but has
no teardown.
3) Fix return values in ct_test.c (introduced by an earlier refactoring,
oops)

Note that for parameterized tests, the index (test vector) usually holds all the
customization, and there should be no need for a separate test
fixture. The CTS test is an exception: it demonstrates how to combine
customization with parameterization.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Prior to TLS1.3 we check that the received record version number is correct.
In TLS1.3 we need to ignore the record version number. This adds a test to
make sure we do it correctly.
Reviewed-by: NRich Salz <rsalz@openssl.org>

The record layer version field must be ignored in TLSv1.3, so we remove the
check when using that version.
Reviewed-by: NRich Salz <rsalz@openssl.org>

We may get failures if we run it in TLS1.3, and it makes no sense anyway
so force TLS1.2
Reviewed-by: NRich Salz <rsalz@openssl.org>

We have one TLS1.3 ciphersuite, but there is a typo in the id that should
be corrected.
Reviewed-by: NRich Salz <rsalz@openssl.org>

We read it later in grow_init_buf(). If CCS is the first thing received in
a flight, then it will use the init_msg from the last flight we received. If
the init_buf has been grown in the meantime then it will point to some
arbitrary other memory location. This is likely to result in grow_init_buf()
attempting to grow to some excessively large amount which is likely to
fail. In practice this should never happen because the only time we receive
a CCS as the first thing in a flight is in an abbreviated handshake. None
of the preceding messages from the server flight would be large enough to
trigger this.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1772)

If zlib-dynamic was given but not --with-zlib-lib, LIBZ was defined to
the empty string.  Instead, give it the default "ZLIB1".
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1772)

VMS only unloads shared libraries at process rundown, so tell the
OpenSSL code so by pretending we linked with -znodelete.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1862)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1856)

Since the local symbol table is looked up before the global symbol
table, 'arch' assigned in the local symbol table of the DCL where MMS
is called would be seen before the 'arch' defined in descrip.mms.
Assigning it to the local symbol table in descrip.mms removes that
issue.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1853)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1849)

The sources for internal tests were sometimes badly formed, assuming
perl variables such as $target{cpuid_asm_src} contains only one file
name.  This change correctly massages all file names in such a
variable.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1850)

This is related to a lack in path merging involding includes of includes
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1846)

The logic around avoiding MULDEF warnings was flawed.  Simplifying it
makes it better.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1846)

The test fixtures are (meant to be) useful for sharing common
setup. Don't bother when we don't have any setup/teardown.

This only addresses simple tests. Parameterized tests (ADD_ALL_TESTS)
will be made more user-friendly in a follow-up.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Clang on Linux seems to catch things that we might miss otherwise.
Also, throw in 'no-deprecated' to make sure we test that as well.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1839)

pqueue_size() now returns a size_t, but the variable that gets returned
was still declared as an int.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Travis is reporting one file at a time shadowed variable warnings where
"read" has been used. This attempts to go through all of libssl and replace
"read" with "readbytes" to fix all the problems in one go.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Give more detail on what constitutes success/failure.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Rename "read" to "readbytes"
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Improvements to style, grammar etc.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Based on review feedback.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Travis was failing in some builds due to a bogus complaint
about uninit variables.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Also document SSL_peek() which was missing from the docs.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Also implement the using of them
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>