libssl code uses EVP_PKEY_get0_EC_KEY() to extract certain basic data
from the EC_KEY.  We replace that with internal EVP_PKEY functions.

This may or may not be refactored later on.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11358)

util/progs.pl depends on the build tree (on configdata.pm,
specifically), so it needs to be run from the build tree.  But why
stop there?  We might as well generate apps/progs.c and apps/progs.h
when building.
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11185)

Since we've now switched to use util/wrap.pl to wrap uninstalled
programs everywhere, there's no need to set the environment variables
OPENSSL_ENGINES and OPENSSL_MODULES globally for the tests.
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/11110)

Remove unused util/process_docs.pl
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10856)

We had all tests run with test/test-runs/ as working directory, and
tests cleaned up after themselves...  which is well and good, until
you want to have a look at what went wrong when a complex test fails,
and you have to recreate everything it does manually.

To remedy this, we have OpenSSL::Test create the result directory
dynamically (and cleaning it up first if it's already there) and let
the test recipe have that as working directory.

Test recipes are now encouraged to name their diverse output files
uniquely, and not to clean them up, to allow a developer to have a
look at the files that were produced.

With continuous integration that allows this, the result directories
could also be archived and be left as a build artifact.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11080)

Because we generate an increasing number of POD files, some of them
end up in the build tree.  This makes it difficult for find-doc-nits
to work as desired when the build tree is separate from the source
tree.

The best supported way to make it work in such an environment is to
run it from the build tree and let it use the build information from
configdata.pm to find all the POD files.  To make this smooth enough,
we add a function 'files' that returns an array of file names
corresponding to criteria from the caller.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11045)

Add cmd-nits make target.

Listing options should stop when it hits the "parameters" separator.

Add missing .pod.in files to doc/man1/build.info

Tweak find-doc-nits to try openssl-XXX before XXX for POD files and
change an error messavge to be more useful.

Fix the following pages: ca, cms, crl, dgst, enc,
engine, errstr, gendsa, genrsa, list, ocsp, passwd, pkcs7, pkcs12, rand,
rehash, req, rsautil, s_server, speed, s_time,
sess_id, smime, srp, ts, x509.
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NDmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/10873)

Replace "=for openssl foreign manuals" with simpler syntax, it looks
like the "=for openssl ifdef" construct.
Fix some broken L<> links; add some missing foreign references and fixed
some typo's.
The WARNINGS in dhparam referred to non-existant commands so reword it.

Fixes #10109
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10256)

When generating html or manpages from POD files, we used $< or $? to
get the file name to process.  It turns out, though, that some make
implementations only define $< with implicit rules, so its expansion
remains empty in explicit rules.  $? is a fine replacement, but only
as long as we have one dependency, so it may cause problems in the
future.

The final solution seems to be to use explicit POD file names
instead.  That leaves no doubts.

Fixes #10817
Reviewed-by: NBen Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/10849)

For some reason, we didn't use some of the possible target attributes
in the Unix Makefile template, and there was a similar but much
smaller lack of use in the Windows makefile template as well.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10753)

Fixes #8322

The leak-checking (and backtrace option, on some platforms) provided
by crypto-mdebug and crypto-mdebug-backtrace have been mostly neutered;
only the "make malloc fail" capability remains.  OpenSSL recommends using
the compiler's leak-detection instead.

The OPENSSL_DEBUG_MEMORY environment variable is no longer used.
CRYPTO_mem_ctrl(), CRYPTO_set_mem_debug(), CRYPTO_mem_leaks(),
CRYPTO_mem_leaks_fp() and CRYPTO_mem_leaks_cb() return a failure code.
CRYPTO_mem_debug_{malloc,realloc,free}() have been removed.  All of the
above are now deprecated.

Merge (now really small) mem_dbg.c into mem.c
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10572)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6236)

The `./pyca-cryptography/.travis/downstream.d` subdirectory that causes the `rm` command to fail (albeit harmlessly, but with a warning from `make` nonetheless).

>rm -f `find . -name '*.d' \! -name '.*' -print`
>rm: cannot remove './pyca-cryptography/.travis/downstream.d': Is a directory
>make: [Makefile:1910: clean] Error 1 (ignored)

Exclude directories from being matched by the `find` commands.

CLA: trivial
Reviewed-by: NPatrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10264)

Fixes #8961
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8990)

For files GENERATEd from templates (.in files), any perl module (.pm
file) that the file depends on will automatically be used.

This means that these two lines:

    GENERATE[foo]=foo.in
    DEPEND[foo]=whatever.pm

will emit this command in a Makefile (or corresponding):

    foo: foo.in whatever.pm configdata.pm
    $(PERL) -I. -Ipathto -Mwhatever -Mconfigdata $(SRCDIR)/util/dofile.pl \\
        foo.in > foo

Note that configdata.pm is automatically added, since util/dofile.pl
itself depends on it.
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10162)

Added functionality to use static libraries as source for other
libraries.  When done this way, the target library will use the object
files from the sourced static libraries, making the sourced libraries
work as "containers" for object files.

We also need to make sure that the Unix Makefile template knows how to
deal with shared libraries and modules that depend on static libraries.
That's new situation we haven't had before.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10088)

The build.info grammar's regular expressions were a horrible read.
By assigning certain sub-expressions to variables, we hope to make
it a little more readable.

Also, the handling of build.info attributes is reworked to use a
common function instead of having copies of the same code.

Finally, the attributes are reorganized to specify if they belong with
programs, libraries, modules or scripts.  This will enable more
intricate attribute assignment in changes to come.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10088)

Use err() for find-doc-nits -e output
Doing this meant we could remove the -s flag, so we do so; move
option/help stuff to top of script.
Add a CHANGES entry.
Rename missing to other.syms
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10039)

Filter all output to a new &err() routine, which sets the global
exit status, $status.
Also, fix all subroutine definitions and references to be consistent:
no prototypes, no & before function calls.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
Reviewed-by: NPaul Yang <kaishen.yy@antfin.com>
(Merged from https://github.com/openssl/openssl/pull/9733)

- Make the last argument always be the output file.
- Make the first argument always be the flavour, even if there is no
  flavour (i.e. it might become the empty string).
- Make the next to last argument to be $(PROCESSOR) if that one has a
  value.
- Remaining arguments are C prepropressor arguments.

Perl scripts that should handle this may use the following code:

    $output = pop;
    $flavour = shift;
    if ($ARGV[$#ARGV] eq '386') {
        # Do 386 specific things
    } else {
        # Do whatever else, with the knowledge the @ARGV contains
        # C preprocessor arguments
    }

Some scripts don't care about anything than $output, and that's ok.
Some scripts do care, but handle it a little differently, and that's
ok too (notably, the x86 scripts call asm_init() with the first and
the last argument after having popped $output).
As long as they handle the argument order right, they are going to
be fine.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9884)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9618)

A local 'make clean' did some sweeping removals of files execpt for
the .git directory.  This is a little too sweeping, as other dotted
files might be cleaned away if they happen to match the pattern that's
searched for.

An example is a symlink .dir-locals.el that would keep disappearing if
you build in the source tree and do a make clean...

So we change this to leave all dotted files alone.  Our builds do not
produce such files anyway, so this is a harmless (or rather, less
harmful) change.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9573)

This makes for a cleaner apps/progs.h as well as as cleaner
apps/build.info.

We also break out the type declarations to apps/include/function.h

apps/progs.c and apps/progs.h are NOT regenerated when 'apps' is
disabled.
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9340)

After avoiding OPENSSL_memcmp for EC curve comparison, there are no remaining
uses in the source code.  The function is only defined in an internal header
and thus should be safe to remove for 3.0.0.
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: NDmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/9207)

We create lists of undocumented functions and macros as they are now so
that find-doc-nits can check for newly introduced functions/macros that
are undocumented.

This works in a similar way to the -u and -d options to find-doc-nits.
These count undocumented symbols and print a detailed list of undocumented
symbols repsectively. This commit adds the -v and -e options to restrict
the count/detailed list to newly added undocumented symbols only.

There is also a new -s option that does the same as -e except that it
produces no output if there are no newly undocumented symbols.

We also amend "make doc-nits" to add the -s option which should cause
travis to fail if a PR adds undocumented symbols.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9094)

Not all Unixen know the -v option
Reviewed-by: NTomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9012)

These will be used to point out general OpenSSL modules directory.
ENGINE modules are kept apart for backward compatibility.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8220)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
GH: #8285

CLA: trivial
Reviewed-by: NMatt Caswell <matt@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8226)

Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8162)

The only thing that makes an ENGINE module special is its entry
points.  Other than that, it's a normal dynamically loadable module,
nothing special about it.  This change has us stop pretending anything
else.

We retain using ENGINE as a term for installation, because it's
related to a specific installation directory, and we therefore also
mark ENGINE modules specifically as such with an attribute in the
build.info files.
Reviewed-by: NNicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8147)

Trim trailing whitespace. It doesn't match OpenSSL coding standards,
AFAICT, and it can cause problems with git tooling.

Trailing whitespace remains in test data and external source.
Reviewed-by: NKurt Roeckx <kurt@roeckx.be>
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8092)

We have two classes of scripts to be installed, those that are
installed as "normal" programs, and those that are installed as "misc"
scripts.  These classes are installed in different locations, so the
build file templates must pay attention.

Because we didn't have the tools to indicate what scripts go where, we
had these scripts hard coded in the build template files, with the
maintenance issues that may cause.  Now that we have attributes, those
can be used to classify the installed scripts, and have the build file
templates simply check the attributes to know what's what.

Furthermore, the 'tsget.pl' script exists both as 'tsget.pl' and
'tsget', which is done by installing a symbolic link (or copy).  This
link name is now given through an attribute, which results in even
less hard coding in the Unix Makefile template.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7581)

This means that all PROGRAMS_NO_INST, LIBS_NO_INST, ENGINES_NO_INST
and SCRIPTS_NO_INST are changed to be PROGRAM, LIBS, ENGINES and
SCRIPTS with the associated attribute 'noinst'.
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7581)

Add platform::Unix, which is a generic Unix module to support product
name and extensions functionlity.  However, this isn't quite enough,
as mingw and Cygwin builds are done using the same templates, but
since shared libraries work as on Windows and are named accordingly,
platform::mingw and platform::Cygwin were also added to provide the
necessary tweaks.

This reworks Configurations/unix-Makefile.tmpl to work out product
names in platform::Unix et al terms.  In this one, we currently do
care about the *_extension config attributes, and the modules adapt
accordingly where it matters.

This change also affected crypto/include/internal/dso_conf.h.in, since
the DSO extension is meant to be the same as the short shared library
extension, which isn't '.so' everywhere.

'shared_extension' attributes that had the value
'.so.\$(SHLIB_VERSION_NUMBER)' are removed, platform::Unix provides
an extension where the shared library version number is hard-coded
instead.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/7473)

This was forgotten in a few places

Fixes #7862
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7865)

Since `$config{version}` and `$config{version_num}` were removed
in commit 3a63dbef, the configure output displays an empty
version number string in parentheses instead of the version number.

This pull request fixes that by adding new config variables
`version` and `full_version`, analogous to `OPENSSL_VERSION_STR`
and `OPENSSL_FULL_VERSION_STR`.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7841)

We're strictly use version numbers of the form MAJOR.MINOR.PATCH.
Letter releases are things of days past.

The most central change is that we now express the version number with
three macros, one for each part of the version number:

    OPENSSL_VERSION_MAJOR
    OPENSSL_VERSION_MINOR
    OPENSSL_VERSION_PATCH

We also provide two additional macros to express pre-release and build
metadata information (also specified in semantic versioning):

    OPENSSL_VERSION_PRE_RELEASE
    OPENSSL_VERSION_BUILD_METADATA

To get the library's idea of all those values, we introduce the
following functions:

    unsigned int OPENSSL_version_major(void);
    unsigned int OPENSSL_version_minor(void);
    unsigned int OPENSSL_version_patch(void);
    const char *OPENSSL_version_pre_release(void);
    const char *OPENSSL_version_build_metadata(void);

Additionally, for shared library versioning (which is out of scope in
semantic versioning, but that we still need):

    OPENSSL_SHLIB_VERSION

We also provide a macro that contains the release date.  This is not
part of the version number, but is extra information that we want to
be able to display:

    OPENSSL_RELEASE_DATE

Finally, also provide the following convenience functions:

    const char *OPENSSL_version_text(void);
    const char *OPENSSL_version_text_full(void);

The following macros and functions are deprecated, and while currently
existing for backward compatibility, they are expected to disappear:

    OPENSSL_VERSION_NUMBER
    OPENSSL_VERSION_TEXT
    OPENSSL_VERSION
    OpenSSL_version_num()
    OpenSSL_version()

Also, this function is introduced to replace OpenSSL_version() for all
indexes except for OPENSSL_VERSION:

    OPENSSL_info()

For configuration, the option 'newversion-only' is added to disable all
the macros and functions that are mentioned as deprecated above.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: NMatt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7724)

Reviewed-by: NMatthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
(Merged from https://github.com/openssl/openssl/pull/7696)