Some miscellaneous removal of dead code from apps. Also fix an issue with
error handling with pkcs7.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Some miscellaneous removal of dead code from lib crypto.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

In the probable_prime() function we behave slightly different if the number
of bits we are interested in is <= BN_BITS2 (the num of bits in a BN_ULONG).
As part of the calculation we work out a size_limit as follows:

    size_limit = (((BN_ULONG)1) << bits) - BN_get_word(rnd) - 1;

There is a problem though if bits == BN_BITS2. Shifting by that much causes
undefined behaviour. I did some tests. On my system BN_BITS2 == 64. So I
set bits to 64 and calculated the result of:

    (((BN_ULONG)1) << bits)

I was expecting to get the result 0. I actually got 1! Strangely this...

    (((BN_ULONG)0) << BN_BITS2)

...does equal 0! This means that, on my system at least, size_limit will be
off by 1 when bits == BN_BITS2.

This commit fixes the behaviour so that we always get consistent results.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

The function CRYPTO_128_unwrap_pad uses an 8 byte AIV (Alternative Initial
Value). The least significant 4 bytes of this is placed into the local
variable |ptext_len|. This is done as follows:

    ptext_len = (aiv[4] << 24) | (aiv[5] << 16) | (aiv[6] << 8) | aiv[7];

aiv[4] is an unsigned char, but (aiv[4] << 24) is promoted to a *signed*
int - therefore we could end up shifting into the sign bit and end up with
a negative value. |ptext_len| is a size_t (typically 64-bits). If the
result of the shifts is negative then the upper bits of |ptext_len| will
all be 1.

This commit fixes the issue by explicitly casting to an unsigned int.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Passing a negative value for the "-time" option to s_time results in a seg
fault. This commit fixes it so that time has to be greater than 0.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

The function tls1_PRF counts the number of digests in use and partitions
security evenly between them. There always needs to be at least one digest
in use, otherwise this is an internal error. Add a sanity check for this.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

The function sk_zero is supposed to zero the elements held within a stack.
It uses memset to do this. However it calculates the size of each element
as being sizeof(char **) instead of sizeof(char *). This probably doesn't
make much practical difference in most cases, but isn't a portable
assumption.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Move memory allocation failure checks closer to the site of the malloc in
dgst app. Only a problem if the debug flag is set...but still should be
fixed.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Add some missing checks for memory allocation failures in ca app.
Reviewed-by: NTim Hudson <tjh@openssl.org>

TABLE was always a debugging tool, and permitted everyone to see the
effect of changes in the string-format configs.  The hash-format
configs being much more readable, distributing TABLE becomes much less
necessary.

Being able to produce a TABLE is kept, however, as it still is a
useful debugging tool for configs, what with multi-level inheritance
and all.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Move obviously personal configurations to personal files.

Note: those files should really not be in the main repo at all
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>

As part of this, remove some levitte examples that never were relevant.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Because base templates express inheritance of values, the attribute is
renamed to 'inherit_from', and texts about this talk about 'inheritance(s)'
rather than base templates.

As they were previously implemented, base templates that were listed
together would override one another, the first one acting as defaults for
the next and so on.

However, it was pointed out that a strength of inheritance would be to
base configurations on several templates - for example one for CPU, one
for operating system and one for compiler - and that requires a different
way of combining those templates.  With this change, inherited values
from several inheritances are concatenated by default (keep on reading).

Also, in-string templates with the double-curly syntax are removed,
replaced with the possibility to have a configuration value be a coderef
(i.e. a 'sub { /* your code goes here */ }') that gets the list of values
from all inheritances as the list @_.  The result of executing such a
coderef on a list of values is assumed to become a string.  ANY OTHER
FORM OF VALUE WILL CURRENTLY BREAK.

As a matter of fact, an attribute in the current config with no value is
assumed to have this coderef as value:

    sub { join(' ', @_) }

While we're at it, rename debug-[cl]flags to debug_[cl]flags and
nodebug-[cl]flags to release_[cl]flags.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>

Base templates are templates that are used to inherit from.  They can
loosely be compared with parent class inheritance in object orientation.
They can be used for the same purpose as the variables with multi-field
strings are used in old-style string configurations.

Base templates are declared with the base_templates configuration
attribute, like so:

	"example_target" => {
		base_templates => [ "x86_asm", ... ]
		...
	}

Note: The value of base_templates MUST be an array reference (an array
enclosed in square brackets).

Any configuration target can be used as a base template by another.  It
is also possible to have a target that's a pure template and not meant to
be used directly as a configuration target.  Such a target is marked with
the template configuration attribute, like so:

	"example_template" => {
		template => 1,
		cc => "mycc",
		...
	},

As part of this commit, all variables with multi-field strings have been
translated to pure templates.  The variables currently remain since we
can't expect people to shift to hash table configurations immediately.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Template references are words with double brackets, and refer to the
same field in the target pointed at the the double bracketed word.

For example, if a target's configuration has the following entry:

    'cflags' => '-DFOO {{x86_debug}}'

... then {{x86_debug}} will be replaced with the 'cflags' value from
target 'x86_debug'.

Note: template references are resolved recursively, and circular
references are not allowed
Reviewed-by: NAndy Polyakov <appro@openssl.org>

The reasoning is that configuration strings are hard to read and error
prone, and that a better way would be for them to be key => value hashes.

Configure is made to be able to handle target configuration values as a
string as well as a hash.  It also does the best it can to combine a
"debug-foo" target with a "foo" target, given that they are similar
except for the cflags and lflags values.  The latter are spliced into
options that are common for "debug-foo" and "foo", options that exist
only with "debug-foo" and options that exist only with "foo", and make
them into combinable attributes that holds common cflags, extra cflags
for debuggin and extra cflags for non-debugging configurations.

The next step is to make it possible to have template configurations.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Previously, ASN1_UTCTIME_cmp_time_t would return 1 if s > t, -1 if
s < t, and 0 if s == t.

This behavior was broken in a refactor [0], resulting in the opposite
time comparison behavior.

[0]: 904348a4922333106b613754136305db229475ea

PR#3706
Reviewed-by: NStephen Henson <steve@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>

Missed one file in the #ifdef merge; thanks Kurt.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

Update error messages to say "EC is disabled" these can then be picked up
by mkdef.pl.

Update ordinals.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

It created the cert structure in SSL_CTX or SSL if it was NULL, but they can
never be NULL as the comments already said.
Reviewed-by: NDr. Stephen Henson <steve@openssl.org>

Other curves don't have this problem.
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NEmilia Käsper <emilia@openssl.org>

Td4 and Te4 are arrays of u8. A u8 << int promotes the u8 to an int first then shifts.
If the mathematical result of a shift (as modelled by lhs * 2^{rhs}) is not representable
in an integer, behaviour is undefined. In other words, you can't shift into the sign bit
of a signed integer. Fix this by casting to u32 whenever we're shifting left by 24.

(For consistency, cast other shifts, too.)

Caught by -fsanitize=shift

Submitted by Nick Lewycky (Google)
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Allocate and free ASN.1 string types directly instead of going through
the ASN.1 item code.
Reviewed-by: NRich Salz <rsalz@openssl.org>

According to RFC 5649 section 4.1 step 1) we should not add padding
if plaintext length is multiply of 8 ockets.

This matches pseudo-code in http://dx.doi.org/10.6028/NIST.SP.800-38F
on page 15, section 6.3 KWP, algorithm 5 KWP-AE, step 2.

PR#3675
Reviewed-by: NStephen Henson <steve@openssl.org>
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Remove DECLARE_ASN1_SET_OF and DECLARE_PKCS12_STACK_OF these haven't been
used internally in OpenSSL for some time.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

When printing out an ASN.1 structure if the type is an item template don't
fall thru and attempt to interpret as a primitive type.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NAndy Polyakov <appro@openssl.org>

In the RSA_X931_derive_ex a call to BN_CTX_new is made. This can return
NULL on error. However the return value is not tested until *after* it is
derefed! Also at the top of the function a test is made to ensure that
|rsa| is not NULL. If it is we go to the "err" label. Unfortunately the
error handling code deref's rsa.
Reviewed-by: NRich Salz <rsalz@openssl.org>

If SSL_check_chain is called with a NULL X509 object or a NULL EVP_PKEY
or the type of the public key is unrecognised then the local variable
|cpk| in tls1_check_chain does not get initialised. Subsequently an
attempt is made to deref it (after the "end" label), and a seg fault will
result.
Reviewed-by: NDr. Stephen Henson <steve@openssl.org>

Fixed assorted missing return value checks in c3_cpols.c
Reviewed-by: NRich Salz <rsalz@openssl.org>

The return value from ASN1_STRING_new() was not being checked which could
lead to a NULL deref in the event of a malloc failure. Also fixed a mem
leak in the error path.
Reviewed-by: NRich Salz <rsalz@openssl.org>

The return value from ASN1_STRING_new() was not being checked which could
lead to a NULL deref in the event of a malloc failure. Also fixed a mem
leak in the error path.
Reviewed-by: NRich Salz <rsalz@openssl.org>

The call to asn1_do_adb can return NULL on error, so we should check the
return value before attempting to use it.
Reviewed-by: NRich Salz <rsalz@openssl.org>

ASN1_primitive_new takes an ASN1_ITEM * param |it|. There are a couple
of conditional code paths that check whether |it| is NULL or not - but
later |it| is deref'd unconditionally. If |it| was ever really NULL then
this would seg fault. In practice ASN1_primitive_new is marked as an
internal function in the public header file. The only places it is ever
used internally always pass a non NULL parameter for |it|. Therefore, change
the code to sanity check that |it| is not NULL, and remove the conditional
checking.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Calling EVP_DigestInit_ex which has already had the digest set up for it
should be possible. You are supposed to be able to pass NULL for the type.
However currently this seg faults.
Reviewed-by: NAndy Polyakov <appro@openssl.org>