The gpmapreduce application is an optional install included via
the --enable-mapreduce configure option. The tests were however
still in src/test/regress and unconditionally included in the
ICW schedule, thus causing test failures when mapreduce wasn't
configured.

Move all gpmapreduce tests to co-locate them with the mapreduce
code and only test when configured.

Also, add a dependency on Perl for gpmapreduce in autoconf since
it's a required component.

Updates to configure to use `getversion --short` to generate a version
tag that can be used in `Makefile.global` for packaging etc...

This reverts commit c9eafeb6.

GP_VERSION is used by gppkg and other bits to show what version of GPDB
it is build from.  This is also propogated via gp_config for all
packages and should have a valid value for GP_VERSION.

If GPDB is configured to build with orca, this commit adds checks to
ensure orca libraries are available.

We also chose more representative headers for the checks.

If gpdb is configured to build with orca it requires a Greenplum patched
version of Xerces library. This commit adds an explicit check for the
patched Xerces.

This also gives us an opportunity to be more idiomatic in doing
autoconf: we are replacing the manual manipulation of LIBS in
src/backend/Makefile with `AC_CHECK_LIB` in `configure.in`.

This changes autoconf to enable orca by default. To be clear, the twin
flags of `--enable-orca` and `--disable-orca` are already available,
this commit merely changes the default (and help messages accordingly).
Signed-off-by: NOmer Arap <oarap@pivotal.io>

When splitting the Greenplum and PostgreSQL versioning in autoconf,
the wrong variable was used when building the version string. Also
add a test for it in ICW to catch it in case it would happen again.

There was a lot of unused code in the gpMgmt bash code that hasn't
been running for quite some time. On top of really dead code, some
codepaths were reachable, but useless (like printing a non-existing
version string). Remove the dead code and replace the version print
with working code which pulls the version from the canonical source.

Signed-off-by: NMarbin Tan <mtan@pivotal.io>

- add new configure option `--enable-gpperfmon`
- include gpperfmon libraries into configure
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>
Signed-off-by: NChumki Roy <croy@pivotal.io>
Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>
Signed-off-by: NC.J. Jameson <cjameson@pivotal.io>

Rather than relying on always having a full cloned repository when
building, add fallbacks which work for trees with constrained (such
as shallow-cloned trees), or no, history. When the tagged commit is
reachable, use git describe, else rely on the version string in
configure since that's required to always be there regardless of
distribution mode. If we have git tree then include the HEAD sha1
as well.

While there, clean up versioning to more clearly separate us from
upstream.

	commit 3cab8bdf
	Author: Tom Lane <tgl@sss.pgh.pa.us>
	Date:   Tue Aug 19 19:17:40 2008 +0000

	    Remove -Winline from the default set of CFLAGS for gcc.  It's gotten much
	    too noisy to be useful as of gcc 4.3, and we were never really doing
	    anything about inlining warnings anyway.

To avoid having to duplicate all the flags passed to CC also for CXX, also
apply CFLAGS to the C++ compiler. Not all of CFLAGS might be applicable
to C++ code, however, so construct CXXFLAGS from CFLAGS, by testing each
flag to see if it also works with CXX.

By default, this adds -Wall and a bunch of other flags to the C++ command
line.

Starting with ORCA version 2.2, there's a gpos/config.h file that contains
flags describing the compile-time options used to build the ORCA library.
Those flags affect binary compatibility, so it's important that e.g. if
ORCA was built with GPOS_DEBUG, the code that uses it (src/backend/gpopt
in this case) is also built with GPOS_DEBUG. Use the new gpos/config.h
for that, instead of deriving them ourselves and hoping that we reach the
same conclusions as whoever built ORCA.

This requires ORCA v2.2, so update releng.mk to download that.

Summary:
* Properly propagate the C++ compiler from autoconf
* Disable aggressive loop optimizations in GCC 4.8+

This commit is in the spirit of postgres/postgres@105f3ef4 , but done
for C++ code so that the GPORCA translator can properly build when the
compiler is GCC 4.8+

Fixes greenplum-db/gpdb#1186 .

TL;DR, here is the additional context on what specific to GPORCA translator:

GCC 6.2.0 compiler (which is required for codegen) got aggressive loop optimization to remove loop when enumerating variable length array.

In code `CTranslatorRelcacheToDXL.cpp`, we have a loop to enumerate all
the attributes of an index:

```
INT iAttno = pgIndex->indkey.values[ul];
```

Where `values[ul]` is of following data structure:

```
typedef struct
{
...
	int2		values[1];		/* VARIABLE LENGTH ARRAY */
} int2vector;					/* VARIABLE LENGTH STRUCT */
```

This is standard way of dealing with variable length array in Postgres. However, optimizer
thinking there is only 1 element, hence removed the loop.

This caused wrong result when using GPDB with GPORCA. The repro is
```
-- multi-column index
create table index_test (a int, b int, c int, d int, e int,
  constraint index_test_pkey PRIMARY KEY (a, b, c, d));

insert into index_test select i,i%2,i%3,i%4,i%5 from generate_series(1,100) i;

-- expecting using index scan with index cond: c=5
explain select * from index_test where c = 5;
```

When compile with GCC 6.2.0, the `index scan` is not selected, but a `table scan` is chosen
instead. The root cause is the compiler removed the loop, and only keep the first column.

The fix is to add $CXXFLAG auto detection to add the
`-fno-aggressive-loop-optimizations` for $CXX compilers if that version of compiler supporting
such optimization. Hence to avoid having the above issue for gpopt component.

In commit 71450d7f, we added code to inform
suitably-intelligent compilers that ereport() doesn't return if the elevel
is ERROR or higher.  This patch extends that to elog(), and also fixes a
double-evaluation hazard that the previous commit created in ereport(),
as well as reducing the emitted code size.

The elog() improvement requires the compiler to support __VA_ARGS__, which
should be available in just about anything nowadays since it's required by
C99.  But our minimum language baseline is still C89, so add a configure
test for that.

The previous commit assumed that ereport's elevel could be evaluated twice,
which isn't terribly safe --- there are already counterexamples in xlog.c.
On compilers that have __builtin_constant_p, we can use that to protect the
second test, since there's no possible optimization gain if the compiler
doesn't know the value of elevel.  Otherwise, use a local variable inside
the macros to prevent double evaluation.  The local-variable solution is
inferior because (a) it leads to useless code being emitted when elevel
isn't constant, and (b) it increases the optimization level needed for the
compiler to recognize that subsequent code is unreachable.  But it seems
better than not teaching non-gcc compilers about unreachability at all.

Lastly, if the compiler has __builtin_unreachable(), we can use that
instead of abort(), resulting in a noticeable code savings since no
function call is actually emitted.  However, it seems wise to do this only
in non-assert builds.  In an assert build, continue to use abort(), so that
the behavior will be predictable and debuggable if the "impossible"
happens.

These changes involve making the ereport and elog macros emit do-while
statement blocks not just expressions, which forces small changes in
a few call sites.

Andres Freund, Tom Lane, Heikki Linnakangas

Conflicts with GPDB codebase:
	config/c-compiler.m4
	configure
	configure.in
	contrib/cube/cubescan.l
	contrib/seg/segscan.l
	src/include/pg_config.h.in
	src/include/pg_config.h.win32

This attempts to clean up the autoconf script a bit and follow the
upstream division of generic code in config/ with the actual lookup
configuration in configure.in. Also updated our installation to rely
on a more modern version of autoconf by backporting parts of upstream
commit 7cc514ac. This commit consist of:

  * Decouple --enable-codegen and --with-codegen-prefix to not
    silently ignore prefixes if the enable flag isn't passed.
	Emit a warning if configuring prefixes without codegen. Also
	moves --with-codegen-prefix to require an argument since
	--with-codegen-prefix without an argument is likely to hide
	either a scripting bug or a misunderstanding from the user
  * Move program checks for cmake and apr-1-config to programs.m4
    and allow for path overrides and ensure to use the resolved
	path when invoking cmake for --enable-codegen
  * Propagate the apr-1-config flags and objects to where used via
    Makefile.global rather than performing another lookup
  * Remove check for unused arguments since autoconf does that
    automatically since 2.63
  * Remove backported fseeko handling since that isn't relevant
    for modern autoconf versions
  * Minor help output tidying and spelling fixes

Fixed a typo in the --with-codegen-prefix description and corrected
spelling in the CMake check error message. No functional changes.

Since we discover the apr-1-config binary during the configure stage
we might as well simplify the gpfdist Makefile by using it during
building. Also renames the APR_CONFIG variable to avoid colliding
with $APR_CONFIG using the gpAux makefile.

Squashed commit of the following:

commit db752c0c916f36479b0c5049c671538565294c25
Author: Nikos Armenatzoglou <narmenatzoglou@gmail.com>
Date:   Thu Mar 24 14:01:38 2016 -0700

    Adding check to avoid unit test failures when sizeof(long) = 8

commit 1c39c58279cf73af92811ed1b2e6f2bc22d01414
Author: Shreedhar Hardikar <shardikar@pivotal.io>
Date:   Thu Mar 17 19:42:47 2016 +0000

    --with-codegen-prefix instead of CMAKE_PREFIX_PATH

commit 77978e92444580bc4e84dc3f6b3e366a09e92dba
Author: Shreedhar Hardikar <shardikar@pivotal.io>
Date:   Thu Mar 10 14:57:48 2016 +0000

    Make codegen work with GPDB build system and other minor fixes

      * Update CMakeLists to build SUBSYS.o
      * Update CMakeLists to build release build by default
      * Update CMakeLists to add tests in a loop.
      * Add codegen and cmake specific .gitignore
      * Update license and READMEs
      * Move codegen utils related header files back into src/backend/codegen/include/codegen
Signed-off-by: NNikos Armenatzoglou <nikos.armenatzoglou@gmail.com>

commit 1b61c12f4eb082f49a66c28957c22810e98a5074
Author: Shreedhar Hardikar <shardikar@pivotal.io>
Date:   Wed Mar 16 21:06:28 2016 +0000

    Remove codegen specific gtest source tree and use the one in gpAux.
Signed-off-by: NNikos Armenatzoglou <nikos.armenatzoglou@gmail.com>

commit 13bb2dcf3465be0ef240e316a149b2d8a125a9e9
Author: Shreedhar Hardikar <shardikar@pivotal.io>
Date:   Thu Mar 10 02:51:29 2016 +0000

    Set up GPDB to use codegen utilities

      * Move in codegen utility source and include files to GPDB specific locations.
      * Configure codegen with --enable-codegen and use gpcodegen namespace
Signed-off-by: NNikos Armenatzoglou <nikos.armenatzoglou@gmail.com>

commit 9a4e40f931f71cb7a82e15b51745944116bc10ab
Author: Craig Chasseur <spoonboy42@gmail.com>
Date:   Mon Nov 2 09:10:37 2015 -0800

    Introduce codegen utilties into GPDB.

    Support library for integrating LLVM code-generation and runtime
    compilation with Clang in a larger codebase.
Signed-off-by: NNikos Armenatzoglou <nikos.armenatzoglou@gmail.com>

We do not need to link all of the gpfdist libraries to
the backend. Make sure that gpfdist is the only one who uses
libyaml, libevent, and libapr-1 for now.

This adds a new switch to autoconf, --enable-mapreduce, to connect
the GPMapreduce extension to the build. If set to on the mapreduce
extension in gpAux/extensions will be built and installed.  As a
side effect this will properly clean and distclean for gpfdist in
gpAux/extensions as well. Autoconf checks for libyaml since it's
a depedency for GPMapreduce.

This is the first stab at gpmapreduce, once in we can start moving
gpmapreduce back into src/bin and move the mapreduce test suite in
under gpAux/extensions.

Commit fc6aedc2 added autoconf checks for
those libraries, but the AC_CHECK_LIB macros have the side-effect of adding
-lDDBoost and -emcconnect to LIBS, which means that they will be linked in
all binaries. That's not what we want, they should only be linked into the
few binaries that actually use them.

This hopefully fixes the build failure seen on some platforms.

Neither ddboost nor emcconnect had proper library or heaer file
checks and were not present in the pg_config.h.

Make the list of options in --help line up a little better and set
proper detailed descriptions on all options. Also set output text
on all options during the ./configure operation.

This will make it so that we can build gpfdist with or without
OpenSSL when doing the configure.
* Print a message that SSL is not supported if built without OpenSSL

As gpfdist will now be part of the gpdb build by default, we may just
treat it as part of gpdb core.
* Integrate gpfdist configure file to the top level configure and
  changed necessary files to accomadate the changes.
* Modify travis.yml for gpfdist structural changes
* Update gpfdist.c to use GP_VERSION from pg_config.h

Fix warning issues from transform.c
* There were many prototype warnings that was being mentioned.
  Added a headerfile for missing prototype warnings, also included the
  structs into the headerfile.

Replace src/port/glob.c
* src/port/glob.c is dead code at the moment and can be safely replaced by
  the windows changes from gpfdist/port

Check for libyaml:
* disable transformations for gpfdist if not found.

* Add configuration option for gpfdist
use --disable-gpfdist to turn off gpfdist build

Enable to run extension regression tests from top level makefile

* Add installcheck gpfdist into extensions
* Add check for enterprise targeted builds
* Pass apr config to gpfdist
* Add JAVA_HOME check

Authors:
Marbin Tan and Shujie Shang

Heikki suggested that we backport atomic operation from PostgreSQL 9.5 to
fix this

    commit b64d92f1
    Author: Andres Freund <andres@anarazel.de>
    Date:   Thu Sep 25 23:49:05 2014 +0200

    Add a basic atomic ops API abstracting away platform/architecture details.

    Several upcoming performance/scalability improvements require atomic
    operations. This new API avoids the need to splatter compiler and
    architecture dependent code over all the locations employing atomic
    ops.

    For several of the potential usages it'd be problematic to maintain
    both, a atomics using implementation and one using spinlocks or
    similar. In all likelihood one of the implementations would not get
    tested regularly under concurrency. To avoid that scenario the new API
    provides a automatic fallback of atomic operations to spinlocks. All
    properties of atomic operations are maintained. This fallback -
    obviously - isn't as fast as just using atomic ops, but it's not bad
    either. For one of the future users the atomics ontop spinlocks
    implementation was actually slightly faster than the old purely
    spinlock using implementation. That's important because it reduces the
    fear of regressing older platforms when improving the scalability for
    new ones.

    The API, loosely modeled after the C11 atomics support, currently
    provides 'atomic flags' and 32 bit unsigned integers. If the platform
    efficiently supports atomic 64 bit unsigned integers those are also
    provided.

    To implement atomics support for a platform/architecture/compiler for
    a type of atomics 32bit compare and exchange needs to be
    implemented. If available and more efficient native support for flags,
    32 bit atomic addition, and corresponding 64 bit operations may also
    be provided. Additional useful atomic operations are implemented
    generically ontop of these.

    The implementation for various versions of gcc, msvc and sun studio have
    been tested. Additional existing stub implementations for
    * Intel icc
    * HUPX acc
    * IBM xlc
    are included but have never been tested. These will likely require
    fixes based on buildfarm and user feedback.

    As atomic operations also require barriers for some operations the
    existing barrier support has been moved into the atomics code.

    Author: Andres Freund with contributions from Oskari Saarenmaa
    Reviewed-By: Amit Kapila, Robert Haas, Heikki Linnakangas and Álvaro Herrera
    Discussion: CA+TgmoYBW+ux5-8Ja=Mcyuy8=VXAnVRHp3Kess6Pn3DMXAPAEA@mail.gmail.com,
        20131015123303.GH5300@awork2.anarazel.de,
        20131028205522.GI20248@awork2.anarazel.de

This patch replaces all occurrences of pgsql-bugs@postgresql.org with
bugs@greenplum.org. Pointers to the pgsql-bugs mailinglist in the code
remain as they re. Regression and BugBuster tests are also patched.

This patch merges the PostgreSQL 9.5 implementation of SSE4.2 into GPDB.
The SSE4.2 implementation was lifted right out of PostgreSQL without
change to make merging later PostgreSQL releases easier.

Update win32 configuration to support SSE4.2 runtime checks

This change was pulled from "src/include/pg_config.h.win32" from the
commits below.

configure.in changes, determining if cpu instruction for perfomring runtime
checks are availible, and moving some of the code to port SSE4.2 came from the
following PostgreSQL commits:

commit 3dc2d62d
Author: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date:   Tue Apr 14 17:05:03 2015 +0300

    Use Intel SSE 4.2 CRC instructions where available.

    Modern x86 and x86-64 processors with SSE 4.2 support have special
    instructions, crc32b and crc32q, for calculating CRC-32C. They greatly
    speed up CRC calculation.

    Whether the instructions can be used or not depends on the compiler and the
    target architecture. If generation of SSE 4.2 instructions is allowed for
    the target (-msse4.2 flag on gcc and clang), use them. If they are not
    allowed by default, but the compiler supports the -msse4.2 flag to enable
    them, compile just the CRC-32C function with -msse4.2 flag, and check at
    runtime whether the processor we're running on supports it. If it doesn't,
    fall back to the slicing-by-8 algorithm. (With the common defaults on
    current operating systems, the runtime-check variant is what you get in
    practice.)

    Abhijit Menon-Sen, heavily modified by me, reviewed by Andres Freund.

commit b4eb2d16
Author: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date:   Tue Apr 14 19:56:00 2015 +0300
	On gcc and clang, the _mm_crc32_u8 and _mm_crc32_u64 intrinsics are not
	defined at all, when not building with -msse4.2. But on icc, they are.
	So we cannot assume that if those intrinsics are defined, we can always use
	them safely, we might still need the runtime check.

	To fix, check if the __SSE_4_2__ preprocessor symbol is defined. That's
	supposed to be defined only when the compiler is targeting a processor that
	has SSE 4.2 support.

	Per buildfarm members fulmar and okapi.

There are no translations for Greenplum-specific strings, so it seems like
a disservice to allow anyone to build with --enable-nls.

The PostgreSQL translation files were a bit of a hodge-podge too; they
seemed to be from PostgreSQL 8.4.1, but we still carried the bunch of
incomplete translations that were removed in PostgreSQL 8.4.1. Let's not
pretend that there's any sense in them, and just remove all .po files. If
we want to make NLS to work properly in the future, we can bring them back,
using some sensible versions. Another option would've been to replace them
with the REL8_2_0 versions, which would possibly have made merging with
upstream slightly easier, as you wouldn't get merge conflicts on the
removed files, but those conflicts are easy to deal with. Also, by removing
the files completely, we don't need to decide what to do with the
references to pgsql-bugs@postgresql.org in them (we're trying to replace
all those references with bugs@greenplum.org).

Update config to include gpopt and gpos header files.

This reverts commit c742462e.