Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4700)

Fixes: #5899
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5911)

Almost all *alloc failures now set an error code.
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5842)

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Conceptually, this is a squashed version of:

    Revert "Address feedback"

    This reverts commit 75551e07.

and

    Revert "Add CRYPTO_thread_glock_new"

    This reverts commit ed6b2c79.

But there were some intervening commits that made neither revert apply
cleanly, so instead do it all as one shot.

The crypto global locks were an attempt to cope with the awkward
POSIX semantics for pthread_atfork(); its documentation (the "RATIONALE"
section) indicates that the expected usage is to have the prefork handler
lock all "global" locks, and the parent and child handlers release those
locks, to ensure that forking happens with a consistent (lock) state.
However, the set of functions available in the child process is limited
to async-signal-safe functions, and pthread_mutex_unlock() is not on
the list of async-signal-safe functions!  The only synchronization
primitives that are async-signal-safe are the semaphore primitives,
which are not really appropriate for general-purpose usage.

However, the state consistency problem that the global locks were
attempting to solve is not actually a serious problem, particularly for
OpenSSL.  That is, we can consider four cases of forking application
that might use OpenSSL:

(1) Single-threaded, does not call into OpenSSL in the child (e.g.,
the child calls exec() immediately)

For this class of process, no locking is needed at all, since there is
only ever a single thread of execution and the only reentrancy is due to
signal handlers (which are themselves limited to async-signal-safe
operation and should not be doing much work at all).

(2) Single-threaded, calls into OpenSSL after fork()

The application must ensure that it does not fork() with an unexpected
lock held (that is, one that would get unlocked in the parent but
accidentally remain locked in the child and cause deadlock).  Since
OpenSSL does not expose any of its internal locks to the application
and the application is single-threaded, the OpenSSL internal locks
will be unlocked for the fork(), and the state will be consistent.
(OpenSSL will need to reseed its PRNG in the child, but that is
an orthogonal issue.)  If the application makes use of locks from
libcrypto, proper handling for those locks is the responsibility of
the application, as for any other locking primitive that is available
for application programming.

(3) Multi-threaded, does not call into OpenSSL after fork()

As for (1), the OpenSSL state is only relevant in the parent, so
no particular fork()-related handling is needed.  The internal locks
are relevant, but there is no interaction with the child to consider.

(4) Multi-threaded, calls into OpenSSL after fork()

This is the case where the pthread_atfork() hooks to ensure that all
global locks are in a known state across fork() would come into play,
per the above discussion.  However, these "calls into OpenSSL after
fork()" are still subject to the restriction to async-signal-safe
functions.  Since OpenSSL uses all sorts of locking and libc functions
that are not on the list of safe functions (e.g., malloc()), this
case is not currently usable and is unlikely to ever be usable,
independently of the locking situation.  So, there is no need to
go through contortions to attempt to support this case in the one small
area of locking interaction with fork().

In light of the above analysis (thanks @davidben and @achernya), go
back to the simpler implementation that does not need to distinguish
"library-global" locks or to have complicated atfork handling for locks.
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/5089)

Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4816)

Since return is inconsistent, I removed unnecessary parentheses and
unified them.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4541)

Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/4294)

cryptilib.h is the second.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4188)

Removed e_os.h from all bar three headers (apps/apps.h crypto/bio/bio_lcl.h and
ssl/ssl_locl.h).

Added e_os.h into the files that need it now.

Directly reference internal/nelem.h when required.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4188)

This is especially harmful since OPENSSL_cleanup() has already called
the RAND cleanup function
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3137)

It's called with 0 when it's already locked, with 1 when it's not.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>

GH: #1500

Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>

GH: #1500

Certain functions are automatically called during auto-deinit in order
to deallocate resources. However, if we have never entered a function which
marks lib crypto as inited then they never get called. This can happen if
the user only ever makes use of a small sub-set of functions that don't hit
the auto-init code.

This commit ensures all such resources deallocated by these functions also
init libcrypto when they are initially allocated.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NBen Laurie <ben@openssl.org>

That way, we have a way to check if the init function was successful
or not.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/996)

There is a preference for suffixes to indicate that a function is internal
rather than prefixes. Note: the suffix is only required to disambiguate
internal functions and public symbols with the same name (but different
case)
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

There was a lot of naming inconsistency, so we try and standardise on
one form.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

ENGINE_cleanup() should not be called expicitly - we should leave
auto-deinit to clean this up instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Remove usage of CRYPTO_LOCK_ENGINE
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Add utility macros REF_ASSERT_NOT and REF_PRINT_COUNT
This is also RT 4181
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Rebased and merged by me, with Ben's approval.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NBen Laurie <ben@openssl.org>

... related to engine_ref_debug macro.
Signed-off-by: NRich Salz <rsalz@akamai.com>
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

This was done by the following
        find . -name '*.[ch]' | /tmp/pl
where /tmp/pl is the following three-line script:
        print unless $. == 1 && m@/\* .*\.[ch] \*/@;
        close ARGV if eof; # Close file to reset $.

And then some hand-editing of other files.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

Add CRYPTO_free_ex_index (for shared libraries)
Unify and complete the documentation for all "ex_data" API's and objects.
Replace xxx_get_ex_new_index functions with a macro.
Added an exdata test.
Renamed the ex_data internal datatypes.
Reviewed-by: NMatt Caswell <matt@openssl.org>

Continuing from previous commit ensure our style is consistent for malloc
return checks.
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

There are many places (nearly 50) where we malloc and then memset.
Add an OPENSSL_zalloc routine to encapsulate that.
(Missed one conversion; thanks Richard)
Also fixes GH328
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Just as with the OPENSSL_malloc calls, consistently use sizeof(*ptr)
for memset and memcpy.  Remove needless casts for those functions.
For memset, replace alternative forms of zero with 0.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

For a local variable:
        TYPE *p;
Allocations like this are "risky":
        p = OPENSSL_malloc(sizeof(TYPE));
if the type of p changes, and the malloc call isn't updated, you
could get memory corruption.  Instead do this:
        p = OPENSSL_malloc(sizeof(*p));
Also fixed a few memset() calls that I noticed while doing this.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Don't check for NULL before calling free functions. This gets:
        ERR_STATE_free
        ENGINE_free
        DSO_free
        CMAC_CTX_free
        COMP_CTX_free
        CONF_free
        NCONF_free NCONF_free_data _CONF_free_data
        A sk_free use within OBJ_sigid_free
        TS_TST_INFO_free (rest of TS_ API was okay)
        Doc update for UI_free (all uses were fine)
        X509V3_conf_free
        X509V3_section_free
        X509V3_string_free
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Following ANSI C rules, remove the casts from calls to
OPENSSL_malloc and OPENSSL_realloc.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

and ENGINE is destroyed.

(Also improve util/ck_errf.pl script, and occasionally
fix source code formatting.)

tree. This further reduces header interdependencies, and makes some
associated cleanups.

of libcrypto, then it is possible that when they are loaded they will share
the same static data as the loading application/library. This means it will
be too late to set memory/ERR/ex_data/[etc] callbacks, but entirely
unnecessary to try. This change puts a static variable in the core ENGINE
code (contained in libcrypto) and a function returning a pointer to it. If
the loaded ENGINE's return value from this function matches the loading
application/library's return value - they share static data. If they don't
match, the loaded ENGINE has its own copy of libcrypto's static data and so
the callbacks need to be set.

Also, although 0.9.7 hasn't been released yet, it's clear this will
introduce a binary incompatibility between dynamic ENGINEs built for 0.9.7
and 0.9.8 (though others probably exist already from EC_*** hooks and
what-not) - so the version control values are correspondingly bumped.

essentially overwrites itself with the new ENGINE, with the exception of
reference counts, ex_data structures, and other 'admin' elements. However
if the new ENGINE doesn't populate certain elements, there's the risk of
the "dynamic" ENGINE's elements showing through - the "cmd_defns" were just
one of the possibilities. This implements a more comprehensive cleanup.