Implementation experience has shown that the original plan for async wait
fds was too simplistic. Originally the async logic created a pipe internally
and user/engine code could then get access to it via API calls. It is more
flexible if the engine is able to create its own fd and provide it to the
async code.

Another issue is that there can be a lot of churn in the fd value within
the context of (say) a single SSL connection leading to continually adding
and removing fds from (say) epoll. It is better if we can provide some
stability of the fd value across a whole SSL connection. This is
problematic because an engine has no concept of an SSL connection.

This commit refactors things to introduce an ASYNC_WAIT_CTX which acts as a
proxy for an SSL connection down at the engine layer.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Fixes github issue 463.  Building the app without OPENSSL_NO_SOCK
isn't supported, so only do OPENSSL_NO_OCSP.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Add -DBIO_DEBUG to --strict-warnings.
Remove comments about outdated debugging ifdef guards.
Remove md_rand ifdef guarding an assert; it doesn't seem used.
Remove the conf guards in conf_api since we use OPENSSL_assert, not assert.
For pkcs12 stuff put OPENSSL_ in front of the macro name.
Merge TLS_DEBUG into SSL_DEBUG.
Various things just turned on/off asserts, mainly for checking non-NULL
arguments, which is now removed: camellia, bn_ctx, crypto/modes.
Remove some old debug code, that basically just printed things to stderr:
  DEBUG_PRINT_UNKNOWN_CIPHERSUITES, DEBUG_ZLIB, OPENSSL_RI_DEBUG,
  RL_DEBUG, RSA_DEBUG, SCRYPT_DEBUG.
Remove OPENSSL_SSL_DEBUG_BROKEN_PROTOCOL.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

On some platforms, the implementation is such that a signed char
triggers a warning when used with is*() functions.  On others, the
behavior is outright buggy when presented with a char that happens
to get promoted to a negative integer.

The safest thing is to cast the char that's used to an unsigned char.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

Also fix option processing in pkeyutl to allow use of (formerly)
"out-of-order" switches that were needless implementation limitations.

Handle documented "ENGINE" form with -keyform and -peerform.

Better handling of OPENSSL_NO_ENGINE and OPENSSL_NO_RSA.

RT2018
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRichard Levitte <levitte@openssl.org>

Tell open() O_BINARY on VMS doesn't make sense, as it's possible to
use more precise file attributes.  However, if we're still going to
fdopen() it in binary mode, we must set the fd in binary context.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Reviewed-by: NRich Salz <rsalz@openssl.org>

Rename BUF_{strdup,strlcat,strlcpy,memdup,strndup,strnlen}
to OPENSSL_{strdup,strlcat,strlcpy,memdup,strndup,strnlen}
Add #define's for the old names.
Add CRYPTO_{memdup,strndup}, called by OPENSSL_{memdup,strndup} macros.
Reviewed-by: NTim Hudson <tjh@openssl.org>

s_server already had the ability to wait on an async file descriptor. This
adds it to s_client too.
Reviewed-by: NRich Salz <rsalz@openssl.org>

continue on from previous commits but in the apps directory
Reviewed-by: NKurt Roeckx <kurt@openssl.org>

RT#4081
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

For those command line options that take the verification options
-CApath and -CAfile, if those options are absent then the default path or
file is used instead. It is not currently possible to specify *no* path or
file at all. This change adds the options -no-CApath and -no-CAfile to
specify that the default locations should not be used to all relevant
applications.
Reviewed-by: NAndy Polyakov <appro@openssl.org>

If the field separator isn't specified through -nameopt then use
XN_FLAG_SEP_CPLUS_SPC instead of printing nothing and returing an error.

PR#2397
Reviewed-by: NTim Hudson <tjh@openssl.org>

This takes away a build failure in some cases.
Reviewed-by: NTim Hudson <tjh@openssl.org>

If the output to stdout or the input from stdin is meant to be binary,
it's deeply unsetting to get the occasional LF converted to CRLF or
the other way around.  If someone happens to forget to redirect stdin
or stdout, they will get gibberish anyway, line ending conversion will
not change that.

Therefore, let's not have dup_bio_* decide unilaterally what mode the
BIO derived from stdin and stdout, and rather let the app decide by
declaring the intended format.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

The different apps had the liberty to decide whether they would open their
input and output files in binary mode or not, which could be confusing if
two different apps were handling the same type of file in different ways.

The solution is to centralise the decision of low level file organisation,
and that the apps would use a selection of formats to state the intent of
the file.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Remove support for RSA_NET and Netscape key format (-keyform n).

Also removed documentation of SGC.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

When generating a private key, try to make the output file be readable
only by the owner.  Put it in CHANGES file since it might be noticeable.

Add "int private" flag to apps that write private keys, and check that it's
set whenever we do write a private key.  Checked via assert so that this
bug (security-related) gets fixed.  Thanks to Viktor for help in tracing
the code-paths where private keys are written.
Reviewed-by: NViktor Dukhovni <viktor@openssl.org>

The module loading feature got broken a while ago, so restore it, but
have it a bit more explicit this time around.
Reviewed-by: NStephen Henson <steve@openssl.org>

Create app_load_config(), a routine to load config file.  Remove the
"always load config" from the main app.  Change the places that used to
load config to call the new common routine.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Given the pervasive nature of TLS extensions it is inadvisable to run
OpenSSL without support for them. It also means that maintaining
the OPENSSL_NO_TLSEXT option within the code is very invasive (and probably
not well tested). Therefore it is being removed.
Reviewed-by: NRich Salz <rsalz@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

For the various string-compare routines (strcmp, strcasecmp, str.*cmp)
use "strcmp()==0" instead of "!strcmp()"
Reviewed-by: NTim Hudson <tjh@openssl.org>

Compiling OpenSSL code with MSVC and /W4 results in a number of warnings.
One category of warnings is particularly interesting - C4701 (potentially
uninitialized local variable 'name' used). This warning pretty much means
that there's a code path which results in uninitialized variables being used
or returned. Depending on compiler, its options, OS, values in registers
and/or stack, the results can be nondeterministic. Cases like this are very
hard to debug so it's rational to fix these issues.

This patch contains a set of trivial fixes for all the C4701 warnings (just
initializing variables to 0 or NULL or appropriate error code) to make sure
that deterministic values will be returned from all the execution paths.

RT#3835
Signed-off-by: NMatt Caswell <matt@openssl.org>

Matt's note: All of these appear to be bogus warnings, i.e. there isn't
actually a code path where an unitialised variable could be used - its just
that the compiler hasn't been able to figure that out from the logic. So
this commit is just about silencing spurious warnings.
Reviewed-by: NRich Salz <rsalz@openssl.org>

Fix error in WIN32_rename() introduced by commit b4faea50.
Reviewed-by: NRich Salz <rsalz@openssl.org>

For a local variable:
        TYPE *p;
Allocations like this are "risky":
        p = OPENSSL_malloc(sizeof(TYPE));
if the type of p changes, and the malloc call isn't updated, you
could get memory corruption.  Instead do this:
        p = OPENSSL_malloc(sizeof(*p));
Also fixed a few memset() calls that I noticed while doing this.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Reviewed-by: NTim Hudson <tjh@openssl.org>

After the finale, the "real" final part. :)  Do a recursive grep with
"-B1 -w [a-zA-Z0-9_]*_free" to see if any of the preceeding lines are
an "if NULL" check that can be removed.
Reviewed-by: NTim Hudson <tjh@openssl.org>

Don't check for NULL before calling free functions. This gets:
        ERR_STATE_free
        ENGINE_free
        DSO_free
        CMAC_CTX_free
        COMP_CTX_free
        CONF_free
        NCONF_free NCONF_free_data _CONF_free_data
        A sk_free use within OBJ_sigid_free
        TS_TST_INFO_free (rest of TS_ API was okay)
        Doc update for UI_free (all uses were fine)
        X509V3_conf_free
        X509V3_section_free
        X509V3_string_free
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Don't check for NULL before calling OPENSSL_free
Reviewed-by: NRichard Levitte <levitte@openssl.org>

This gets BN_.*free:
    BN_BLINDING_free BN_CTX_free BN_FLG_FREE BN_GENCB_free
    BN_MONT_CTX_free BN_RECP_CTX_free BN_clear_free BN_free BUF_MEM_free

Also fix a call to DSA_SIG_free to ccgost engine and remove some #ifdef'd
dead code in engines/e_ubsec.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Don't check for NULL before calling free function.  This gets:
        NAME_CONSTRAINTS_free GENERAL_SUBTREE_free ECDSA_METHOD_free
        JPAKE_CTX_free OCSP_REQ_CTX_free SCT_free SRP_VBASE_free
        SRP_gN_free SRP_user_pwd_free TXT_DB_free
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Add OPENSSL_clear_free which merges cleanse and free.
(Names was picked to be similar to BN_clear_free, etc.)
Removed OPENSSL_freeFunc macro.
Fixed the small simple ones that are left:
        CRYPTO_free CRYPTO_free_locked OPENSSL_free_locked
Reviewed-by: NRichard Levitte <levitte@openssl.org>

No point in proceeding if you're out of memory.  So change
*all* OPENSSL_malloc calls in apps to use the new routine which
prints a message and exits.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Don't check for NULL before calling a free routine.  This gets X509_.*free:
    x509_name_ex_free X509_policy_tree_free X509_VERIFY_PARAM_free
    X509_STORE_free X509_STORE_CTX_free X509_PKEY_free
    X509_OBJECT_free_contents X509_LOOKUP_free X509_INFO_free
Reviewed-by: NRichard Levitte <levitte@openssl.org>

Do not check for NULL before calling a free routine.  This addresses:
    ASN1_BIT_STRING_free ASN1_GENERALIZEDTIME_free ASN1_INTEGER_free
    ASN1_OBJECT_free ASN1_OCTET_STRING_free ASN1_PCTX_free ASN1_SCTX_free
    ASN1_STRING_clear_free ASN1_STRING_free ASN1_TYPE_free
    ASN1_UTCTIME_free M_ASN1_free_of
Reviewed-by: NRichard Levitte <levitte@openssl.org>