for specialized Montgomery ladder implementations

PR #6009 and #6070 replaced the default EC point multiplication path for
prime and binary curves with a unified Montgomery ladder implementation
with various timing attack defenses (for the common paths when a secret
scalar is feed to the point multiplication).
The newly introduced default implementation directly used
EC_POINT_add/dbl in the main loop.

The scaffolding introduced by this commit allows EC_METHODs to define a
specialized `ladder_step` function to improve performances by taking
advantage of efficient formulas for differential addition-and-doubling
and different coordinate systems.

- `ladder_pre` is executed before the main loop of the ladder: by
  default it copies the input point P into S, and doubles it into R.
  Specialized implementations could, e.g., use this hook to transition
  to different coordinate systems before copying and doubling;
- `ladder_step` is the core of the Montgomery ladder loop: by default it
  computes `S := R+S; R := 2R;`, but specific implementations could,
  e.g., implement a more efficient formula for differential
  addition-and-doubling;
- `ladder_post` is executed after the Montgomery ladder loop: by default
  it's a noop, but specialized implementations could, e.g., use this
  hook to transition back from the coordinate system used for optimizing
  the differential addition-and-doubling or recover the y coordinate of
  the result point.

This commit also renames `ec_mul_consttime` to `ec_scalar_mul_ladder`,
as it better corresponds to what this function does: nothing can be
truly said about the constant-timeness of the overall execution of this
function, given that the underlying operations are not necessarily
constant-time themselves.
What this implementation ensures is that the same fixed sequence of
operations is executed for each scalar multiplication (for a given
EC_GROUP), with no dependency on the value of the input scalar.
Co-authored-by: NSohaib ul Hassan <soh.19.hassan@gmail.com>
Co-authored-by: NBilly Brumley <bbrumley@gmail.com>
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6690)

Fix the NULL check lack in a different way that is more compatible with
non-NULL branch.  Refer #6632

Also mark and pop the error stack instead of clearing all errors when something
goes awry in CONF_get_number.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6643)

Currently if you encounter application data while waiting for a
close_notify from the peer, and you have called SSL_shutdown() then
you will get a -1 return (fatal error) and SSL_ERROR_SYSCALL from
SSL_get_error(). This isn't accurate (it should be SSL_ERROR_SSL) and
isn't persistent (you can call SSL_shutdown() again and it might then work).

We change this into a proper fatal error that is persistent.
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/6340)

Implement support for stateful TLSv1.3 tickets, and use them if
SSL_OP_NO_TICKET is set.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6563)

This commit implements coordinate blinding, i.e., it randomizes the
representative of an elliptic curve point in its equivalence class, for
prime curves implemented through EC_GFp_simple_method,
EC_GFp_mont_method, and EC_GFp_nist_method.

This commit is derived from the patch
https://marc.info/?l=openssl-dev&m=131194808413635 by Billy Brumley.

Coordinate blinding is a generally useful side-channel countermeasure
and is (mostly) free. The function itself takes a few field
multiplicationss, but is usually only necessary at the beginning of a
scalar multiplication (as implemented in the patch). When used this way,
it makes the values that variables take (i.e., field elements in an
algorithm state) unpredictable.

For instance, this mitigates chosen EC point side-channel attacks for
settings such as ECDH and EC private key decryption, for the
aforementioned curves.

For EC_METHODs using different coordinate representations this commit
does nothing, but the corresponding coordinate blinding function can be
easily added in the future to extend these changes to such curves.
Co-authored-by: NNicola Tuveri <nic.tuv@gmail.com>
Co-authored-by: NBilly Brumley <bbrumley@gmail.com>
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6501)

Use EVP_PKEY_set_alias_type to access
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6443)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6443)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6511)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6217)

Only applies to algorithms that support it. Both raw private and public
keys can be obtained for X25519, Ed25519, X448, Ed448. Raw private keys
only can be obtained for HMAC, Poly1305 and SipHash

Fixes #6259
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6394)

There were a large number of error codes that were unused (probably a
copy&paste from somewhere else). Since these have never been made public
we should remove then and rebuild the error codes.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6386)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6386)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6386)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6386)

OpenSSL 1.1.0 made the X509_LOOKUP_METHOD structure opaque, so
applications that were previously able to define a custom lookup method
are not able to be ported.

This commit adds getters and setters for each of the current fields of
X509_LOOKUP_METHOD, along with getters and setters on several associated
opaque types (such as X509_LOOKUP and X509_OBJECT).
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6152)

This reverts commit a6f5b116.

The EVP_PKEY_sign() function is intended for pre-hashed input which is
not supported by our EdDSA implementation.

See the discussion in PR 5880
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6284)

which is used for ASN1_STRING_print_ex*() and X509_NAME_print_ex*().
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6105)

Add a script to find them in the future
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/6103)

Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6085)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6071)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5837)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5948)

Adding support for these operations for the EdDSA implementations
makes pkeyutl usable for signing/verifying for these algorithms.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5880)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5886)

There are two undocumented DSA parameter generation options available in
the genpkey command line app:
dsa_paramgen_md and dsa_paramgen_q_bits.

These can also be accessed via the EVP API but only by using
EVP_PKEY_CTX_ctrl() or EVP_PKEY_CTX_ctrl_str() directly. There are no
helper macros for these options.

dsa_paramgen_q_bits sets the length of q in bits (default 160 bits).
dsa_paramgen_md sets the digest that is used during the parameter
generation (default SHA1). In particular the output length of the digest
used must be equal to or greater than the number of bits in q because of
this code:

            if (!EVP_Digest(seed, qsize, md, NULL, evpmd, NULL))
                goto err;
            if (!EVP_Digest(buf, qsize, buf2, NULL, evpmd, NULL))
                goto err;
            for (i = 0; i < qsize; i++)
                md[i] ^= buf2[i];

            /* step 3 */
            md[0] |= 0x80;
            md[qsize - 1] |= 0x01;
            if (!BN_bin2bn(md, qsize, q))
                goto err;

qsize here is the number of bits in q and evpmd is the digest set via
dsa_paramgen_md. md and buf2 are buffers of length SHA256_DIGEST_LENGTH.
buf2 has been filled with qsize bits of random seed data, and md is
uninitialised.

If the output size of evpmd is less than qsize then the line "md[i] ^=
buf2[i]" will be xoring an uninitialised value and the random seed data
together to form the least significant bits of q (and not using the
output of the digest at all for those bits) - which is probably not what
was intended. The same seed is then used as an input to generating p. If
the uninitialised data is actually all zeros (as seems quite likely)
then the least significant bits of q will exactly match the least
significant bits of the seed.

This problem only occurs if you use these undocumented and difficult to
find options and you set the size of q to be greater than the message
digest output size. This is for parameter generation only not key
generation. This scenario is considered highly unlikely and
therefore the security risk of this is considered negligible.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5800)

The GOST engine needs to be loaded before we initialise libssl. Otherwise
the GOST ciphersuites are not enabled. However the SSL conf module must
be loaded before we initialise libcrypto. Otherwise we will fail to read
the SSL config from a config file properly.

Another problem is that an application may make use of both libcrypto and
libssl. If it performs libcrypto stuff first and OPENSSL_init_crypto()
is called and loads a config file it will fail if that config file has
any libssl stuff in it.

This commit separates out the loading of the SSL conf module from the
interpretation of its contents. The loading piece doesn't know anything
about SSL so this can be moved to libcrypto. The interpretation of what it
means remains in libssl. This means we can load the SSL conf data before
libssl is there and interpret it when it later becomes available.

Fixes #5809
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5818)

Fail harshly (in debug builds) when rand_pool_acquire_entropy isn't
delivering the required amount of entropy.  In release builds, this
produces an error with details.

We also take the opportunity to modernise the types used.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5857)

Almost all *alloc failures now set an error code.
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5842)

methods :
 - EVP_PBE_scrypt
 - EVP_PKEY_meth_add0
 - EVP_PKEY_meth_new
 - EVP_PKEY_CTX_dup
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NBernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5783)

Constructed types with a recursive definition (such as can be found in
PKCS7) could eventually exceed the stack given malicious input with
excessive recursion. Therefore we limit the stack depth.

CVE-2018-0739

Credit to OSSFuzz for finding this issue.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4793)

There is a requirements of having access to a live entropy source
which we can't do with the default callbacks. If you need prediction
resistance you need to set up your own callbacks that follow the
requirements of NIST SP 800-90C.
Reviewed-by: NDr. Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
GH: #5402

This commit adds a new api RAND_DRBG_set_defaults() which sets the
default type and flags for new DRBG instances. See also #5576.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5632)

Fixes #4403

This commit moves the internal header file "internal/rand.h" to
<openssl/rand_drbg.h>, making the RAND_DRBG API public.
The RAND_POOL API remains private, its function prototypes were
moved to "internal/rand_int.h" and converted to lowercase.

Documentation for the new API is work in progress on GitHub #5461.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5462)

Renamed to EVP_PKEY_new_raw_private_key()/EVP_new_raw_public_key() as per
feedback.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5520)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5520)

Not all algorithms will support this, since their keys are not a simple
block of data. But many can.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5520)

Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5616)

With the current mechanism, old cipher strings that used to work in 1.1.0,
may inadvertently disable all TLSv1.3 ciphersuites causing connections to
fail. This is confusing for users.

In reality TLSv1.3 are quite different to older ciphers. They are much
simpler and there are only a small number of them so, arguably, they don't
need the same level of control that the older ciphers have.

This change splits the configuration of TLSv1.3 ciphers from older ones.
By default the TLSv1.3 ciphers are on, so you cannot inadvertently disable
them through your existing config.

Fixes #5359
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5392)

Add functions that will do the work of assigning certificate, privatekey
and chain certs to an SSL or SSL_CTX. If no privatekey is given, use the
publickey. This will permit the keys to pass validation for both ECDSA
and RSA. If a private key has already been set for the certificate, it
is discarded. A real private key can be set later.

This is an all-or-nothing setting of these parameters. Unlike the
SSL/SSL_CTX_use_certificate() and SSL/SSL_CTX_use_PrivateKey() functions,
the existing cert or privatekey is not modified (i.e. parameters copied).
This permits the existing cert/privatekey to be replaced.

It replaces the sequence of:
* SSL_use_certificate()
* SSL_use_privatekey()
* SSL_set1_chain()
And may actually be faster, as multiple checks are consolidated.

The private key can be NULL, if so an ENGINE module needs to contain the
actual private key that is to be used.

Note that ECDH (using the certificate's ECDSA key) ciphers do not work
without the private key being present, based on how the private key is
used in ECDH. ECDH does not offer PFS; ECDHE ciphers should be used instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/1130)