This adds the possibility to exclude files by regexp in util/copy.pl

Partial fix for #3254
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6303)

LINK can outsmart itself and choose to not update export .lib upon
corresponding .dll re-link. Since dependency is between .lib and all
.obj-s, re-compilation of any .obj makes NMAKE relink .dll and all
.exe-s over and over...
Reviewed-by: NRich Salz <rsalz@openssl.org>

deletion of *.exp files in krb5 sub-module.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6186)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5958)

Purpose of build_all_generated is to execute all the rules that require
perl, so that one can copy the tree to system with compiler but without
perl. This commit removes last dependencies on perl.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5929)

Computing the value of the GENERATED variable in the build file
templates is somewhat overcomplicated, and because of possible
duplication errors, changes are potentially error prone.

Looking more closely at how this list is determined, it can be
observed that the exact list of files to check is consistently
available in all the values found in the %unified_info tables
'depends', 'sources' and 'shared_sources', and all that's needed is to
filter those values so only those present as keys in the 'generate'
table are left.

This computation is also common for all build files, so due to its
apparent complexity, we move it to common0.tmpl, with the result left
in a global variable (@generated), to be consumed by all build file
templates.

common0.tmpl is included among the files to process when creating
build files, but unlike common.tmpl, it comes first of all.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5930)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5900)

The LIBZ macro definition was already quoted in BASE_windows, then got
quotified once more in windows-makefile.tmpl.  That's a bit too much
quotations, ending up with the compiler being asked to define the
macro |"LIBZ=\"ZLIB1\""| (no, not the macro LIBZ with the value
"ZLIB1").  This is solved by removing the extra quoting in
BASE_windows.

Along with this, change the quotation of macro definitions and include
file specification, so we end up with things like -I"QuotedPath" and
-D"Macro=\"some weird value\"" rather than "-IQuotedPath" and
"-DMacro=\"some weird value\"".

Fixes #5827
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5833)

'nmake clean' was leaving some artefacts behind.
Reviewed-by: NRichard Levitte <levitte@openssl.org>

In the end, it's more efficient to only have one perl instance (that
loads configdata.pm) dealing with dependency files than running one
(that still loads configdata.pm) for each such file.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5631)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5635)

It seems that only gcc -MMD produces dependency files that are "sane"
for our needs.  For all other methods, some post processing is needed:

- 'makedepend' (Unix) insists that object files are located in the
  same spot as the source file.
- 'cl /Zs /showIncludes' (Visual C) has "Note: including file: " where
  we'd like to see the object.
- 'CC/DECC' (VMS) insists that the object file is located in the
  current directory, i.e. it strips away all directory information.

So far, we've managed this (except for the VMS case) with individual
uncommented perl command lines directly in the build file template.
We're now collecting these diverse hacks into one perl script that
takes an argument to tell what kind of input to expect and that
massages whatever it gets on STDIN and outputs the result on STDOUT.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5591)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5608)

With the help of the perl script util/add-depends.pl, which takes all
its information directly from configdata.pm, the dependency adding
procedure can be streamlined for all support platforms.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5606)

$target{build_scheme} consists of fixed number of elements with 3rd
element denoting VC install-path "flavour", i.e. where to install things.
Instead of looking at 3rd, let's look at last. This allows to override
flavour from template in a simple way.

Configurations/10-main.conf: define generic "flavour" in VC-common
template. Since VC-W32 was the only recognized "flavour", remove
"flavour" definitions from all targets/templates, but VC-WIN32. And
rename VC-W32 to VC-WOW.
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5502)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5560)

With the support of "make variables" comes the possibility for the
user to override them.  However, we need to make a difference between
defaults that we use (and that should be overridable by the user) and
flags that are crucial for building OpenSSL (should not be
overridable).

Typically, overridable flags are those setting optimization levels,
warnings levels, that kind of thing, while non-overridable flags are,
for example, macros that indicate aspects of how the config target
should be treated, such as L_ENDIAN and B_ENDIAN.

We do that differentiation by allowing upper case attributes in the
config targets, named exactly like the "make variables" we support,
and reserving the lower case attributes for non-overridable project
flags.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5534)

Rely on the build.info constructor to do the right thing.

Fixes #5500
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5501)

With this, we introduce the make variable 'libdir', which differs from
'LIBDIR' not only in casing, but also by being the absolute path to
the library installation directory.  This variable is intentionally
compatible with the GNU coding standards.

When --libdir is given an absolute path, it is considered as a value
according to GNU coding standards, and the variables LIBDIR and libdir
will be this:

    LIBDIR=
    libdir=/absolute/path

When --libdir is given a relative path (just the name of the desired
library directory), or not given at all, it is considered as a
"traditional" OpenSSL value, and the variables LIBDIR and libdir will
be this:

    LIBDIR=relativepath
    libdir=$(INSTALLTOP)/$(LIBDIR)

Fixes #5398
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5446)

The reason for this is that some of the C flags affect built in macros
that we may depend on.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5436)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5247)

The make variables LIB_CFLAGS, DSO_CFLAGS and so on were used in
addition to CFLAGS and so on.  This works without problem on Unix and
Windows, where options with different purposes (such as -D and -I) can
appear anywhere on the command line and get accumulated as they come.
This is not necessarely so on VMS.  For example, macros must all be
collected and given through one /DEFINE, and the same goes for
inclusion directories (/INCLUDE).

So, to harmonize all platforms, we repurpose make variables starting
with LIB_, DSO_ and BIN_ to be all encompassing variables that
collects the corresponding values from CFLAGS, CPPFLAGS, DEFINES,
INCLUDES and so on together with possible config target values
specific for libraries DSOs and programs, and use them instead of the
general ones everywhere.

This will, for example, allow VMS to use the exact same generators for
generated files that go through cpp as all other platforms, something
that has been impossible to do safely before now.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5357)

Instead of having the knowledge of the exact flags to run the C
preprocessor only and have it output on standard output in the deeper
recesses of the build file template, make it a config parameter, or
rely on build CPP in value ('$(CC) -E' on Unix).
Reviewed-by: NPaul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/5356)

It was a bit absurd to have this being specially handled in the build
file templates, especially that we have the 'includes' attribute.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5296)

The slash should be there according to Microsoft documentation,
see https://msdn.microsoft.com/en-us/library/7cafx990.aspx

Fixes #5277
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5278)

Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5197)

For proper escaping, we need the direct perl variable values, not a
make variable reference.
Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5196)

Reviewed-by: NTim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5196)

Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5185)

Support the following "make variables":

AR              (GNU compatible)
ARFLAGS         (GNU Compatible)
AS              (GNU Compatible)
ASFLAGS         (GNU Compatible)
CC              (GNU Compatible)
CFLAGS          (GNU Compatible)
CXX             (GNU Compatible)
CXXFLAGS        (GNU Compatible)
CPP             (GNU Compatible)
CPPFLAGS        (GNU Compatible)
CPPDEFINES      List of CPP macro definitions.  Alternative for -D
CPPINCLUDES     List of CPP inclusion directories.  Alternative for -I
HASHBANGPERL    Perl invocation to be inserted after '#!' in public
                perl scripts.
LDFLAGS         (GNU Compatible)
LDLIBS          (GNU Compatible)
RANLIB          Program to generate library archive index
RC              Program to manipulate Windows resources
RCFLAGS         Flags for $(RC)
RM              (GNU Compatible)

Setting one of these overrides the corresponding data from our config
targets.  However, flags given directly on the configuration command
line are additional, and are therefore added to the flags coming from
one of the variables above or the config target.

Fixes #2420
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5177)

C preprocessor flags get separated from C flags, which has the
advantage that we don't get loads of macro definitions and inclusion
directory specs when linking shared libraries, DSOs and programs.

This is a step to add support for "make variables" when configuring.
Reviewed-by: NTim Hudson <tjh@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5177)

Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5108)

Only when building the main shared libraries

Fixes #5075
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5077)

Remove some config attributes that just duplicate values that are
already there in other attributes.

Remove the special runs of mkdef.pl and mkrc.pl from build file
templates, as these are now done via GENERATE statements in
build.info.

Remove all references to ordinal files from build file templates, as
these are now treated via the GENERATE statements in build.info.

Also remove -shared flags and similar that are there in shared-info.pl
anyway.  (in the case of darwin, it's mandatory, as -bundle and
-dynamiclib don't mix)
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4840)

Because this also includes handling all sorts of non-object files when
linking a program, shared library or DSO, this also includes allowing
general recognition of files such as .res files (compiled from .rc
files), or .def / .map / .opt files (for export and possibly
versioning of public symbols only).

This does mean that there's a tangible change for all build file
templates: they must now recognise and handle the `.o` extension,
which is used internally to recognise object files internally.  This
extension was removed by common.tmpl before this change, but would
mean that the platform specific templates wouldn't know if "foo.map"
was originally "foo.map.o" (i.e. an object file in its own right) or
"foo.map" (an export definition file that should be treated as such,
not as an object file).

For the sake of simplifying things, we also modify util/mkdef.pl to
produce .def (Windows) and .opt (VMS) files that don't need additional
hackery.
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4840)

This will replace the use of Makefile.shared

This also means a small adjustment on how the attributes dso_cflags,
dso_cxxflags and dso_lflags are treated.  They were previously treated
as an extension to shared_cflag, shared_cxxflag and shared_ldflag, but
they should really be regarded as alternatives instead, for example
for darwin, where -dynamiclib is used for shared libraries and -bundle
for DSOs.

We take the opportunity to clean out things that are redundant or
otherwise superfluous (for example the check of GNU ld on platforms
where it never existed).
Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4840)

RESULT_D can be used to provide a separate directory for test results.
Let's use that to separate them from other files.
Reviewed-by: NRich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4507)

This avoids issues that can come with an ending backslash, among other.

Fixes #4458
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4461)

This should fix the recent AppVeyor failures.

[extended tests]
Reviewed-by: NAndy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4171)

Reviewed-by: NAndy Polyakov <appro@openssl.org>
Reviewed-by: NRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4167)