MMXLOG records should only be replayed in standby mode and smgr create
xlog records should not do anything until persistent tables and MMXLOG
records are removed from Greenplum.

Author: Jimmy Yih <jyih@pivotal.io>
Author: Asim Praveen <apraveen@pivotal.io>

There was a hack to exclude guc.c from doing single-line comment (//)
removal when creating mock c files. The hack is no longer needed in
guc.c but is now needed in guc_gp.c since the offending line was moved
to guc_gp.c.

1. When forked command is not blocking, give the output that the pipe
   received to see why it did not block.
2. Fix utility mode quit so that a utility mode connection can be
   restarted when connection is closed by postmaster.
3. Fix ! shell commands by processing -- as part of the shell command
   instead of the start of a SQL comment

Some master processes were calling gpvars_assign_gp_fts_probe_pause
when they should not. This was found when the new stats collecter
process errored out on this function call. We also move it to fts.c
where it will live alongside the other fts code.

Also, check for interrupts in fts sleep loop so that fts probe process
can process SIGHUP properly.

Prior to this commit, the below warnings were occurring with GCC versions 6.2.0
on CentOS 6 and 6.3.0 on Ubuntu 14.
These are actually invalid because the `get_dbinfo()` call will always fill in
those variables but the compiler is not smart enough to detect this.

The code change of this commit has not happened on the latest upstream Postgres
(10.1), so there will be merge conflicts in future, but we decided as a team
in #3871 that it was better to fix the warnings.

In file included from ../../../src/include/postgres.h:53:0,
                 from dbcommands.c:22:
dbcommands.c: In function ‘createdb’:
../../../src/include/utils/elog.h:184:4: error: ‘src_ctype’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
    errfinish rest; \
    ^~~~~~~~~
dbcommands.c:616:11: note: ‘src_ctype’ was declared here
  char    *src_ctype;
           ^~~~~~~~~
In file included from ../../../src/include/postgres.h:53:0,
                 from dbcommands.c:22:
../../../src/include/utils/elog.h:184:4: error: ‘src_collate’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
    errfinish rest; \
    ^~~~~~~~~
dbcommands.c:615:11: note: ‘src_collate’ was declared here
  char    *src_collate;
           ^~~~~~~~~~~
Signed-off-by: NDavid Sharp <dsharp@pivotal.io>

Prior to this commit, the below warnings were occurring on GCC 6.2.0.

The warnings seem valid but unlikely, so we just NULL initialize
logStreamStart, which would most likely still land in the correct error
case.

gcc -O3 -std=gnu99  -Wall -Wmissing-prototypes -Wpointer-arith -Werror=uninitialized -Wendif-labels -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -fno-aggressive-loop-optimizations -Wno-unused-but-set-variable -Wno-address  -fpic -I. -I. -I../../../src/include -D_GNU_SOURCE -I/usr/include/libxml2   -c -o gplibpq.o gplibpq.c
In file included from ../../../src/include/postgres.h:53:0,
                 from gplibpq.c:7:
gplibpq.c: In function 'test_receive_and_verify':
../../../src/include/utils/elog.h:311:3: error: 'logStreamStart.xrecoff' may be used uninitialized in this function [-Werror=maybe-uninitialized]
   elog_finish(elevel, __VA_ARGS__); \
   ^~~~~~~~~~~
gplibpq.c:140:15: note: 'logStreamStart.xrecoff' was declared here
    XLogRecPtr logStreamStart;
               ^~~~~~~~~~~~~~
In file included from ../../../src/include/postgres.h:53:0,
                 from gplibpq.c:7:
../../../src/include/utils/elog.h:311:3: error: 'logStreamStart.xlogid' may be used uninitialized in this function [-Werror=maybe-uninitialized]
   elog_finish(elevel, __VA_ARGS__); \
   ^~~~~~~~~~~
gplibpq.c:140:15: note: 'logStreamStart.xlogid' was declared here
    XLogRecPtr logStreamStart;
               ^~~~~~~~~~~~~~
Signed-off-by: NAmil Khanzada <akhanzada@pivotal.io>

Fixes for:
- PGAC_FUNC_GETTIMOFDAY_1ARG and
- PGAC_FUNC_GETPWUID_R_5ARG

Update the C code for ACX_PTHREAD from the autoconf-archive

The relevant changes come from these commits:
- http://git.savannah.gnu.org/gitweb/?p=autoconf-archive.git;a=commitdiff;h=0728dab771b2a4da8032d5f0ade8870add680b39
- http://git.savannah.gnu.org/gitweb/?p=autoconf-archive.git;a=commitdiff;h=7a6c511965489c38f5d3bef4afe113e3fe11c2ac

> This can be used to mark custom built binaries with an extra version
> string such as a git describe identifier or distribution package release
> version.

From: Oskari Saarenmaa <os@ohmu.fi>
Originally commited by Peter Eisentraut, adapted by Pivotal.
(cherry picked from commit 46328916)

We already had the pointer size check in the upstream order in our
genesis open source commit 6b0e52be , but we seem to have missed the
rest of the rearrangement from upstream.

This commit completes that cherry-pick, to reduce merge diff with
(admittedly much later) commit 46328916 .

> Doesn't seem like a good idea to be doing AC_CHECK_SIZEOF(void *) so much
> earlier than all the other sizeof checks, and it certainly fails to follow
> the order suggested at the file head.  Rearrange.
Author: Tom Lane
Adapted by Pivotal

(cherry picked from commit 075ac80d)

(cherry picked from commit b32a290b)

Harmless, but clearly useless and unintentional.

This resulted in useless extra work during every call of parseRelOptions,
but no bad effects other than that.  Noted by Alvaro.

(Cherry-picked from PostgreSQL commit eb9954e362)

The options should only be registered once. Surprisingly, the duplicates
seem to be harmless, because everything worked, but the accumulation of
more and more reloptions was slowing down tests that created a lot of
tables.

Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

A bad 8.4 merge resolution on my part. We may not support older
server_versions in GPDB's pg_dump, but that doesn't mean we should use
uninitialized memory if we come across one.

error_unsupported_server_version() comes from pg_dump.c; this commit is
a companion to b4c5a618.

We were inconsistent with what's logged and what's not. And some objects
were logged with the GPDB-specific timestamp headers, while others were
not. Make it consistent, by removing the extra logging altogether, making
it consistent with the upstream. The status messages are now only logged in
verbose mode, and there are no timestamps on them. Use 'ts' or similar if
you wish to have timestamps in your logs.

Author: Karen Huddleston <khuddleston@pivotal.io>

It was only used at the end of cdb_dump_agent, shortly before it calls
exit(). There's no need to reset variables or free memory if we're just
about to exit the whole process. pg_dump leaves behind a lot of small
allocations, this was just the tip of the iceberg. Better to be identical
to the upstream, to avoid merge conflicts. (reset() was an awfully generic
name for a function anyway, BTW)

In the passing, also mark buildIndexArray() function as static, like it is
in the upstream.

Commit 8e6aed42 changed the format of the error
messages for reloptions but missed updating the answer files for the default
storage parameters test.

Author: Abhijit Subramanya <asubramanya@pivotal.io>
Author: Taylor Vesely <tvesely@pivotal.io>

* Remove GPHDFS 1.0 and 1.1

* Remove custom ivy repo settings

Merge the next commit from PostgreSQL 8.4, which refactors reloptions.c to
use a table-based parser. We are merging this as a separate commit, because
this needed also refactoring all the GPDB-added reloptions to the new
model.

This adds a new RELOPT_KIND_INTERNAL "relation kind", for TOAST tables and
auxiliary tables of append-only relations, like the AO visimap and block
dir. That will probably be refactored away later in the merge, when we
merge upstream commit 1c855f01, as that commit introduces a new
RELOPT_KIND_TOASTVALUE kind for TOAST tables, instead.

This also includes the new fillRelOptions(), from upstream commit
8ebe1e35 (also from 8.4), because that was very handy in the refactoring
of the GPDB code.

Author: Xiaoran Wang <xiwang@pivotal.io>
Author: Heikki Linnakangas <hlinnakangas@pivotal.io>

"all tuples visible" flag in heap page headers.  The flag update *must*
be applied before calling XLogInsert, but heap_update and the tuple
moving routines in VACUUM FULL were ignoring this rule.  A crash and
replay could therefore leave the flag incorrectly set, causing rows
to appear visible in seqscans when they should not be.  This might explain
recent reports of data corruption from Jeff Ross and others.

In passing, do a bit of editorialization on comments in visibilitymap.c.

(This is a cherry-pick of upstream PostgreSQL commit fedb166549. We bumped
into this bug in the "test_crash_recovery_schema_topology.py" tests in the
concourse pipeline.)

The function DefineForeignRelation referenced in utility.c was removed
in 601c58c3.
Signed-off-by: NTaylor Vesely <tvesely@pivotal.io>

Signed-off-by: NXin Zhang <xzhang@pivotal.io>

This is more portable, as some compliant shells don't understand
"exit -1".

To quote POSIX 2008:

> The exit status shall be n, if specified, except that the behavior is
> unspecified if n is not an unsigned decimal integer or is greater than
> 255.

This commit replaces invocation of `echo -e` with printf.

echo as defined by POSIX only supports -n. "echo -e" is a GNU-ism and
Bashism.  Noticeably /bin/echo -e behaves very differently on macOS vs
on GNU/Linux.

After commit 2b51c16b Greenplum diverged
from the upstream behavior of using `popen`, and instead captures the
stderr of the COPY PROGRAM and prints that out in case of error (read:
non-zero return status).

This means that we no longer specially handle the case of return status
127 and use `wait_result_to_str` to tell the user "command not found".
Instead, we output whatever is in the stderr of the standard shell
/bin/sh.

That means a command-not-found is completely indistinguishable from the
case where the program invoked returning non-zero status and printing
something into its stderr. Moreover, the output of command-not-found is
not portable, and not really controlled by Greenplum/Postgres.

We already have coverage for cases involving returning stderr from
segments and master to the user, and the command not found cases are not
only redundant coverage, they are also brittle.

This commit removes the test cases for command-not-found and updates the
expected test output.

This reverts commit 6fed381d .

After python version updated, gpdb report build problem even after 'make distclean'.
The root cause is shell command broken because multiple python versions directories in below:
    gpMgmt/bin/pythonSrc/subprocess32
    gpMgmt/bin/pythonSrc/PyGreSQL-4.0/build/lib.linux

The new upstream code looks equivalent to the old commented-out code.
Remove the old code, as suggested in the comment.

The answer to the question is "yes, yes we do".

This reverts commit 916ea339.

Linux systems don't seem to mind, probably because time.h gets pulled in
inderectly via some other header. But the Windows job in the pipeline
failed because of this.

With "! <command>", the "set -e" doesn't kick in. As a result, the test
script was incorrectly passing, if one of these connection attempts that
were not supposed to succeed, succeeded.

As mentioned in the FIXME comment, since commit a27addbc, a SIGHUP is
enough to change Kerberos settings. A restart is no longer required. Adjust
test case, it's a lot faster to just reload the config than to restart the
server.