Looks like you can't actually get here with any aggregates or placeholders
in the start/end offsets, or we would've gotten errors.

The uao drop_while_vacuum tests are non deterministic they way they are
currently written. For now we want to remove these tests to make the CI more
stable. The tests will be replaced with a more deterministic version later.

Author: Abhijit Subramanya <asubramanya@pivotal.io>
Author: Taylor Vesely <tvesely@pivotal.io>

These were missed out and need to be ignored.

Author: Jimmy Yih <jyih@pivotal.io>
Author: Asim R Praveen <apraveen@pivotal.io>

When a primary segment first detects mirror is up, primary will
automatically turn on synchronous WAL replication by directly setting
WalSndCtl->sync_standbys_defined to true during FTS probe handling
(bypassing the need to have postmaster reload and Checkpointer process
doing the update). The state will be persisted by inserting
synchronous_standby_names="*" into gp_replication.conf in case of
segment crash.

MMXLOG records should only be replayed in standby mode and smgr create
xlog records should not do anything until persistent tables and MMXLOG
records are removed from Greenplum.

Author: Jimmy Yih <jyih@pivotal.io>
Author: Asim Praveen <apraveen@pivotal.io>

There was a hack to exclude guc.c from doing single-line comment (//)
removal when creating mock c files. The hack is no longer needed in
guc.c but is now needed in guc_gp.c since the offending line was moved
to guc_gp.c.

1. When forked command is not blocking, give the output that the pipe
   received to see why it did not block.
2. Fix utility mode quit so that a utility mode connection can be
   restarted when connection is closed by postmaster.
3. Fix ! shell commands by processing -- as part of the shell command
   instead of the start of a SQL comment

Some master processes were calling gpvars_assign_gp_fts_probe_pause
when they should not. This was found when the new stats collecter
process errored out on this function call. We also move it to fts.c
where it will live alongside the other fts code.

Also, check for interrupts in fts sleep loop so that fts probe process
can process SIGHUP properly.

Prior to this commit, the below warnings were occurring with GCC versions 6.2.0
on CentOS 6 and 6.3.0 on Ubuntu 14.
These are actually invalid because the `get_dbinfo()` call will always fill in
those variables but the compiler is not smart enough to detect this.

The code change of this commit has not happened on the latest upstream Postgres
(10.1), so there will be merge conflicts in future, but we decided as a team
in #3871 that it was better to fix the warnings.

In file included from ../../../src/include/postgres.h:53:0,
                 from dbcommands.c:22:
dbcommands.c: In function ‘createdb’:
../../../src/include/utils/elog.h:184:4: error: ‘src_ctype’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
    errfinish rest; \
    ^~~~~~~~~
dbcommands.c:616:11: note: ‘src_ctype’ was declared here
  char    *src_ctype;
           ^~~~~~~~~
In file included from ../../../src/include/postgres.h:53:0,
                 from dbcommands.c:22:
../../../src/include/utils/elog.h:184:4: error: ‘src_collate’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
    errfinish rest; \
    ^~~~~~~~~
dbcommands.c:615:11: note: ‘src_collate’ was declared here
  char    *src_collate;
           ^~~~~~~~~~~
Signed-off-by: NDavid Sharp <dsharp@pivotal.io>

Prior to this commit, the below warnings were occurring on GCC 6.2.0.

The warnings seem valid but unlikely, so we just NULL initialize
logStreamStart, which would most likely still land in the correct error
case.

gcc -O3 -std=gnu99  -Wall -Wmissing-prototypes -Wpointer-arith -Werror=uninitialized -Wendif-labels -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -fno-aggressive-loop-optimizations -Wno-unused-but-set-variable -Wno-address  -fpic -I. -I. -I../../../src/include -D_GNU_SOURCE -I/usr/include/libxml2   -c -o gplibpq.o gplibpq.c
In file included from ../../../src/include/postgres.h:53:0,
                 from gplibpq.c:7:
gplibpq.c: In function 'test_receive_and_verify':
../../../src/include/utils/elog.h:311:3: error: 'logStreamStart.xrecoff' may be used uninitialized in this function [-Werror=maybe-uninitialized]
   elog_finish(elevel, __VA_ARGS__); \
   ^~~~~~~~~~~
gplibpq.c:140:15: note: 'logStreamStart.xrecoff' was declared here
    XLogRecPtr logStreamStart;
               ^~~~~~~~~~~~~~
In file included from ../../../src/include/postgres.h:53:0,
                 from gplibpq.c:7:
../../../src/include/utils/elog.h:311:3: error: 'logStreamStart.xlogid' may be used uninitialized in this function [-Werror=maybe-uninitialized]
   elog_finish(elevel, __VA_ARGS__); \
   ^~~~~~~~~~~
gplibpq.c:140:15: note: 'logStreamStart.xlogid' was declared here
    XLogRecPtr logStreamStart;
               ^~~~~~~~~~~~~~
Signed-off-by: NAmil Khanzada <akhanzada@pivotal.io>

Harmless, but clearly useless and unintentional.

This resulted in useless extra work during every call of parseRelOptions,
but no bad effects other than that.  Noted by Alvaro.

(Cherry-picked from PostgreSQL commit eb9954e362)

The options should only be registered once. Surprisingly, the duplicates
seem to be harmless, because everything worked, but the accumulation of
more and more reloptions was slowing down tests that created a lot of
tables.

A bad 8.4 merge resolution on my part. We may not support older
server_versions in GPDB's pg_dump, but that doesn't mean we should use
uninitialized memory if we come across one.

error_unsupported_server_version() comes from pg_dump.c; this commit is
a companion to b4c5a618.

We were inconsistent with what's logged and what's not. And some objects
were logged with the GPDB-specific timestamp headers, while others were
not. Make it consistent, by removing the extra logging altogether, making
it consistent with the upstream. The status messages are now only logged in
verbose mode, and there are no timestamps on them. Use 'ts' or similar if
you wish to have timestamps in your logs.

Author: Karen Huddleston <khuddleston@pivotal.io>

It was only used at the end of cdb_dump_agent, shortly before it calls
exit(). There's no need to reset variables or free memory if we're just
about to exit the whole process. pg_dump leaves behind a lot of small
allocations, this was just the tip of the iceberg. Better to be identical
to the upstream, to avoid merge conflicts. (reset() was an awfully generic
name for a function anyway, BTW)

In the passing, also mark buildIndexArray() function as static, like it is
in the upstream.

Commit 8e6aed42 changed the format of the error
messages for reloptions but missed updating the answer files for the default
storage parameters test.

Author: Abhijit Subramanya <asubramanya@pivotal.io>
Author: Taylor Vesely <tvesely@pivotal.io>

"all tuples visible" flag in heap page headers.  The flag update *must*
be applied before calling XLogInsert, but heap_update and the tuple
moving routines in VACUUM FULL were ignoring this rule.  A crash and
replay could therefore leave the flag incorrectly set, causing rows
to appear visible in seqscans when they should not be.  This might explain
recent reports of data corruption from Jeff Ross and others.

In passing, do a bit of editorialization on comments in visibilitymap.c.

(This is a cherry-pick of upstream PostgreSQL commit fedb166549. We bumped
into this bug in the "test_crash_recovery_schema_topology.py" tests in the
concourse pipeline.)

The function DefineForeignRelation referenced in utility.c was removed
in 601c58c3.
Signed-off-by: NTaylor Vesely <tvesely@pivotal.io>

Signed-off-by: NXin Zhang <xzhang@pivotal.io>

This is more portable, as some compliant shells don't understand
"exit -1".

To quote POSIX 2008:

> The exit status shall be n, if specified, except that the behavior is
> unspecified if n is not an unsigned decimal integer or is greater than
> 255.

This commit replaces invocation of `echo -e` with printf.

echo as defined by POSIX only supports -n. "echo -e" is a GNU-ism and
Bashism.  Noticeably /bin/echo -e behaves very differently on macOS vs
on GNU/Linux.

After commit 2b51c16b Greenplum diverged
from the upstream behavior of using `popen`, and instead captures the
stderr of the COPY PROGRAM and prints that out in case of error (read:
non-zero return status).

This means that we no longer specially handle the case of return status
127 and use `wait_result_to_str` to tell the user "command not found".
Instead, we output whatever is in the stderr of the standard shell
/bin/sh.

That means a command-not-found is completely indistinguishable from the
case where the program invoked returning non-zero status and printing
something into its stderr. Moreover, the output of command-not-found is
not portable, and not really controlled by Greenplum/Postgres.

We already have coverage for cases involving returning stderr from
segments and master to the user, and the command not found cases are not
only redundant coverage, they are also brittle.

This commit removes the test cases for command-not-found and updates the
expected test output.

This reverts commit 6fed381d .

The new upstream code looks equivalent to the old commented-out code.
Remove the old code, as suggested in the comment.

The answer to the question is "yes, yes we do".

This reverts commit 916ea339.

Linux systems don't seem to mind, probably because time.h gets pulled in
inderectly via some other header. But the Windows job in the pipeline
failed because of this.

With "! <command>", the "set -e" doesn't kick in. As a result, the test
script was incorrectly passing, if one of these connection attempts that
were not supposed to succeed, succeeded.

As mentioned in the FIXME comment, since commit a27addbc, a SIGHUP is
enough to change Kerberos settings. A restart is no longer required. Adjust
test case, it's a lot faster to just reload the config than to restart the
server.

As noted in the FIXME comment.

The code that used it was removed in commit f51f2f57, but it left behind
these.

We've seen a lot of failures in the 'sreh' test in the pipeline, like this:

--- 263,269 ----
  FORMAT 'text' (delimiter '|')
  SEGMENT REJECT LIMIT 10000;
  SELECT * FROM sreh_ext;
! ERROR:  connection failed dummy_protocol://DUMMY_LOCATION
  INSERT INTO sreh_target SELECT * FROM sreh_ext;
  NOTICE:  Found 10 data formatting errors (10 or more input rows). Rejected related input data.
  SELECT count(*) FROM sreh_target;

I don't really know, but I'm guessing it could be because it sometimes
takes more than one second for gpfdist to fully start up, if there's a lot
of disk or other activity. Increase the sleep time from 1 to 3 seconds;
we'll see if that helps.

The previous attempt, by priming the internal prepared plan by a dummy
pg_get_viewdef() call, turned out to not be very robust. I think that's
because activity in concurrent tests would sometimes cause the plan to be
invalidated and re-planned.

Disable optimizer_trace_fallback around the pg_get-viewdef() calls instead.

The code looks correct, pronargdefaults is in the fixed part of
Form_pg_proc, so you don't need to use SysCacheGetAttr() to access it.

We have now merged user-defined I/O conversion casts, and
SFRM_Materialize_Random, from PostgreSQL 8.4. We can change the JSON code
that depended on those features to be identical to the upstream code now.