Add a new compression option for append-optimized tables, "zstd". It is
generally faster than zlib or quicklz, and compresses better. Or at least
it can be faster or compress better, if not both at the same time, by
adjusting the compression level. A major advantage of Zstandard is the wide
range of tuning, to choose the trade-off between compression speed and
ratio.

Update documentation to mention "zstd" alongside "zlib" and "quicklz". More
could be done; all the examples still use zlib or quicklz, for example, and
I think we want to emphasize Zstandard more in the docs, over those other
options. But this is the bare minimum to keep the docs factually correct.

Using the new option requires building the server with the libzstd library.
A new --with-zstd option is added for that. The default is to build without
libzstd, for now, but we should probably change the default to be on, after
we have had a chance to update all the buildfarm machines to have libzstd.

Patch by Ivan Leskin, Dmitriy Pavlov, Anton Chevychalov. Test case, docs
changes, and some minor editorialization by Heikki Linnakangas.

Many of the autoconf tests produce warnings, or even compiler errors, on purpose
as they run through the conftest programs. So, treating warning as error should
be last step after all autoconf checks are performed, otherwise false
side-effects happens.

Author: Ashwin Agrawal <aagrawal@pivotal.io>
Author: Xin Zhang <xzhang@pivotal.io>

Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

- Remove appending extra-version to Postgres Version
Signed-off-by: NGoutam Tadi <gtadi@pivotal.io>

Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

Signed-off-by: NGoutam Tadi <gtadi@pivotal.io>

> This can be used to mark custom built binaries with an extra version
> string such as a git describe identifier or distribution package release
> version.

From: Oskari Saarenmaa <os@ohmu.fi>
Originally commited by Peter Eisentraut, adapted by Pivotal.
(cherry picked from commit 46328916)

We already had the pointer size check in the upstream order in our
genesis open source commit 6b0e52be , but we seem to have missed the
rest of the rearrangement from upstream.

This commit completes that cherry-pick, to reduce merge diff with
(admittedly much later) commit 46328916 .

> Doesn't seem like a good idea to be doing AC_CHECK_SIZEOF(void *) so much
> earlier than all the other sizeof checks, and it certainly fails to follow
> the order suggested at the file head.  Rearrange.
Author: Tom Lane
Adapted by Pivotal

(cherry picked from commit 075ac80d)

(cherry picked from commit b32a290b)

Building with PXF was made the default in df1d6ccb,
but the autoconf --help message was missed. Update to match the
default config.

* Enable PXF by default for GPDB builds

* Disable pxf wherever libcurl is disabled

Leverage the core autoconf scaffolding for resolving the dependency
on libcurl. Enabling PXF in autoconf now automatically adds libcurl
as a dependency. Coupled with the recent commit which relaxes the
curl version requirement on macOS, we can remove the library copying
from the PXF makefile as well.

The bzip2 library is only used by the gfile/fstream code, used for external
tables and gpfdist. The usage of bzip2 was in #ifndef WIN32 blocks, so it
was only built on non-Windows systems.

Instead of tying it to the platform, use a proper autoconf check and
HAVE_LIBBZ2 flags. This makes it possible to build gpfdist with bzip2
support on Windows, as well as building without bzip2 on non-Windows
systems. That makes it easier to test the otherwise Windows-only codepaths
on other platforms. --with-libbz2 is still the default, but you can now use
--without-libbz2 if you wish.

I'm sure that some regression tests will fail if you actually build the
server without libbz2, but I'm not going to address that right now. We have
similar problems with other features that are in principle optional, but
cause some regression tests to fail.

Also use "#ifdef HAVE_LIBZ" rather than "#ifndef WIN32" to enable/disable
zlib support in gpfdist. Building the server still fails if you use
--without-zlib, but at least you can build the client programs without
zlib, also on non-Windows systems.

Remove obsolete copy of bzlib.h from the repository while we're at it.

* Use built-in JSON parser for PXF fragments

Instead of relying on a external library, use the built-in JSON
parser in the backend for the PXF fragments parsing. Since this
replaces the current implementation with an event-based callback
parser, the code is more complicated, but dogfooding the parser
that we want extension writers to use is a good thing.

This removes the dependency on json-c from autoconf, and enables
building PXF on Travis for extra coverage.

* Use elog for internal errors, and ereport for user errors

Internal errors where we are interested in source filename should
use elog() which will decorate the error messages automatically
with this information. The connection error is interesting for the
user however, use ereport() instead there.

The perl code in src/test/regress was using a mix of either not
printing the version, printing it wrong (due to us not using CVS
anymore) or using a hardcoded string. Implement a new module for
common test code called GPTest.pm which abstracts this (for now
it's the only thing it does but this might/will change, hence the
name). The module is created by autoconf to make it pull in the
GP_VERSION from there.

While there, simplify the version output in gpdiff which included
the version of the system diff command - somewhat uninteresting
information as it's not something that changes very often and just
cluttered up the output.

This removes the MakeMaker support but since we have no intention
of packaging these programs into a CPAN module it seems pointless
to carry that format around.

* Refactor GPHDFS regression to run for pxf

* remove customized Hadoop home location

* PXF tarball creation inside GPDB pipeline

* Remove legacy directory

* Use enable_pxf instead of with_pxf

Signed-off-by: NJohn Gaskin <Quikling@pivotal.io>

Signed-off-by: NLav Jain <ljain@pivotal.io>
Signed-off-by: NOleksandr Diachenko <odiachenko@pivotal.io>
Signed-off-by: NShivram Mani <smani@pivotal.io>
Signed-off-by: NJohn Gaskin <jgaskin@pivotal.io>
Signed-off-by: NAlexandra Wang <lewang@pivotal.io>

Record the version in configure.in for releases which are not full
Git trees.

Ants Aasma and Jeff Davis

(cherry picked from commit fdea2530)

* Port Postgres TAP SSL tests
Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>

* Add config to enable tap tests
Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>

1. Add enable-tap-tests flag to control tests.
2. Add Perl checking module.
3. Enable tap tests for enterprise build by default.

* Adapt postgres tap tests to gpdb

1. Assume a running GPDB cluster instance(gpdemo), instead of
using temp installation. Remove most node init operation.
Disable environment variable override during test init.

2. Replace node control operation with GPDB counterpart:
start   -> gpstart -a
stop    -> gpstop -a
restart -> gpstop -arf
reload  -> gpstop -u

disable promote, add restart_qd.
restart_qd -> pg_ctl -w -t 3 -D $MASTER_DATA_DIRECTORY

3. Add default server key and certificate for GPDB.

4. Update server setup to work with running gpdemo.

5. Disable SSL alternative names cases.
Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>

Signed-off-by: NAbhijit Subramanya <asubramanya@pivotal.io>

This commit does things below to make developers' life, who are still
using old version C++ compilers, easier.

Add a flag to optionally disable gpcloud compilation.
Add a configure check for C++11 if gpcloud is enabled.
Signed-off-by: NAdam Lee <ali@pivotal.io>
Signed-off-by: NHaozhou Wang <hawang@pivotal.io>

1. Enable windows ccl pipeline job, fix configure options.
2. Remove C++ check when orca is disabled, add needed header check.
3. Remove non-exist client docs from msi bundle.
Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>

The gpmapreduce application is an optional install included via
the --enable-mapreduce configure option. The tests were however
still in src/test/regress and unconditionally included in the
ICW schedule, thus causing test failures when mapreduce wasn't
configured.

Move all gpmapreduce tests to co-locate them with the mapreduce
code and only test when configured.

Also, add a dependency on Perl for gpmapreduce in autoconf since
it's a required component.

Updates to configure to use `getversion --short` to generate a version
tag that can be used in `Makefile.global` for packaging etc...

This reverts commit c9eafeb6.

GP_VERSION is used by gppkg and other bits to show what version of GPDB
it is build from.  This is also propogated via gp_config for all
packages and should have a valid value for GP_VERSION.

If GPDB is configured to build with orca, this commit adds checks to
ensure orca libraries are available.

We also chose more representative headers for the checks.

If gpdb is configured to build with orca it requires a Greenplum patched
version of Xerces library. This commit adds an explicit check for the
patched Xerces.

This also gives us an opportunity to be more idiomatic in doing
autoconf: we are replacing the manual manipulation of LIBS in
src/backend/Makefile with `AC_CHECK_LIB` in `configure.in`.

This changes autoconf to enable orca by default. To be clear, the twin
flags of `--enable-orca` and `--disable-orca` are already available,
this commit merely changes the default (and help messages accordingly).
Signed-off-by: NOmer Arap <oarap@pivotal.io>

When splitting the Greenplum and PostgreSQL versioning in autoconf,
the wrong variable was used when building the version string. Also
add a test for it in ICW to catch it in case it would happen again.

There was a lot of unused code in the gpMgmt bash code that hasn't
been running for quite some time. On top of really dead code, some
codepaths were reachable, but useless (like printing a non-existing
version string). Remove the dead code and replace the version print
with working code which pulls the version from the canonical source.

Signed-off-by: NMarbin Tan <mtan@pivotal.io>

- add new configure option `--enable-gpperfmon`
- include gpperfmon libraries into configure
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>
Signed-off-by: NChumki Roy <croy@pivotal.io>
Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>
Signed-off-by: NC.J. Jameson <cjameson@pivotal.io>