A bad 8.4 merge resolution on my part. We may not support older
server_versions in GPDB's pg_dump, but that doesn't mean we should use
uninitialized memory if we come across one.

error_unsupported_server_version() comes from pg_dump.c; this commit is
a companion to b4c5a618.

We were inconsistent with what's logged and what's not. And some objects
were logged with the GPDB-specific timestamp headers, while others were
not. Make it consistent, by removing the extra logging altogether, making
it consistent with the upstream. The status messages are now only logged in
verbose mode, and there are no timestamps on them. Use 'ts' or similar if
you wish to have timestamps in your logs.

Author: Karen Huddleston <khuddleston@pivotal.io>

It was only used at the end of cdb_dump_agent, shortly before it calls
exit(). There's no need to reset variables or free memory if we're just
about to exit the whole process. pg_dump leaves behind a lot of small
allocations, this was just the tip of the iceberg. Better to be identical
to the upstream, to avoid merge conflicts. (reset() was an awfully generic
name for a function anyway, BTW)

In the passing, also mark buildIndexArray() function as static, like it is
in the upstream.

Commit 8e6aed42 changed the format of the error
messages for reloptions but missed updating the answer files for the default
storage parameters test.

Author: Abhijit Subramanya <asubramanya@pivotal.io>
Author: Taylor Vesely <tvesely@pivotal.io>

* Remove GPHDFS 1.0 and 1.1

* Remove custom ivy repo settings

Merge the next commit from PostgreSQL 8.4, which refactors reloptions.c to
use a table-based parser. We are merging this as a separate commit, because
this needed also refactoring all the GPDB-added reloptions to the new
model.

This adds a new RELOPT_KIND_INTERNAL "relation kind", for TOAST tables and
auxiliary tables of append-only relations, like the AO visimap and block
dir. That will probably be refactored away later in the merge, when we
merge upstream commit 1c855f01, as that commit introduces a new
RELOPT_KIND_TOASTVALUE kind for TOAST tables, instead.

This also includes the new fillRelOptions(), from upstream commit
8ebe1e35 (also from 8.4), because that was very handy in the refactoring
of the GPDB code.

Author: Xiaoran Wang <xiwang@pivotal.io>
Author: Heikki Linnakangas <hlinnakangas@pivotal.io>

"all tuples visible" flag in heap page headers.  The flag update *must*
be applied before calling XLogInsert, but heap_update and the tuple
moving routines in VACUUM FULL were ignoring this rule.  A crash and
replay could therefore leave the flag incorrectly set, causing rows
to appear visible in seqscans when they should not be.  This might explain
recent reports of data corruption from Jeff Ross and others.

In passing, do a bit of editorialization on comments in visibilitymap.c.

(This is a cherry-pick of upstream PostgreSQL commit fedb166549. We bumped
into this bug in the "test_crash_recovery_schema_topology.py" tests in the
concourse pipeline.)

The function DefineForeignRelation referenced in utility.c was removed
in 601c58c3.
Signed-off-by: NTaylor Vesely <tvesely@pivotal.io>

Signed-off-by: NXin Zhang <xzhang@pivotal.io>

This is more portable, as some compliant shells don't understand
"exit -1".

To quote POSIX 2008:

> The exit status shall be n, if specified, except that the behavior is
> unspecified if n is not an unsigned decimal integer or is greater than
> 255.

This commit replaces invocation of `echo -e` with printf.

echo as defined by POSIX only supports -n. "echo -e" is a GNU-ism and
Bashism.  Noticeably /bin/echo -e behaves very differently on macOS vs
on GNU/Linux.

After commit 2b51c16b Greenplum diverged
from the upstream behavior of using `popen`, and instead captures the
stderr of the COPY PROGRAM and prints that out in case of error (read:
non-zero return status).

This means that we no longer specially handle the case of return status
127 and use `wait_result_to_str` to tell the user "command not found".
Instead, we output whatever is in the stderr of the standard shell
/bin/sh.

That means a command-not-found is completely indistinguishable from the
case where the program invoked returning non-zero status and printing
something into its stderr. Moreover, the output of command-not-found is
not portable, and not really controlled by Greenplum/Postgres.

We already have coverage for cases involving returning stderr from
segments and master to the user, and the command not found cases are not
only redundant coverage, they are also brittle.

This commit removes the test cases for command-not-found and updates the
expected test output.

This reverts commit 6fed381d .

After python version updated, gpdb report build problem even after 'make distclean'.
The root cause is shell command broken because multiple python versions directories in below:
    gpMgmt/bin/pythonSrc/subprocess32
    gpMgmt/bin/pythonSrc/PyGreSQL-4.0/build/lib.linux

The new upstream code looks equivalent to the old commented-out code.
Remove the old code, as suggested in the comment.

The answer to the question is "yes, yes we do".

This reverts commit 916ea339.

Linux systems don't seem to mind, probably because time.h gets pulled in
inderectly via some other header. But the Windows job in the pipeline
failed because of this.

With "! <command>", the "set -e" doesn't kick in. As a result, the test
script was incorrectly passing, if one of these connection attempts that
were not supposed to succeed, succeeded.

As mentioned in the FIXME comment, since commit a27addbc, a SIGHUP is
enough to change Kerberos settings. A restart is no longer required. Adjust
test case, it's a lot faster to just reload the config than to restart the
server.

As noted in the FIXME comment.

The code that used it was removed in commit f51f2f57, but it left behind
these.

We've seen a lot of failures in the 'sreh' test in the pipeline, like this:

--- 263,269 ----
  FORMAT 'text' (delimiter '|')
  SEGMENT REJECT LIMIT 10000;
  SELECT * FROM sreh_ext;
! ERROR:  connection failed dummy_protocol://DUMMY_LOCATION
  INSERT INTO sreh_target SELECT * FROM sreh_ext;
  NOTICE:  Found 10 data formatting errors (10 or more input rows). Rejected related input data.
  SELECT count(*) FROM sreh_target;

I don't really know, but I'm guessing it could be because it sometimes
takes more than one second for gpfdist to fully start up, if there's a lot
of disk or other activity. Increase the sleep time from 1 to 3 seconds;
we'll see if that helps.

The previous attempt, by priming the internal prepared plan by a dummy
pg_get_viewdef() call, turned out to not be very robust. I think that's
because activity in concurrent tests would sometimes cause the plan to be
invalidated and re-planned.

Disable optimizer_trace_fallback around the pg_get-viewdef() calls instead.

The code looks correct, pronargdefaults is in the fixed part of
Form_pg_proc, so you don't need to use SysCacheGetAttr() to access it.

We have now merged user-defined I/O conversion casts, and
SFRM_Materialize_Random, from PostgreSQL 8.4. We can change the JSON code
that depended on those features to be identical to the upstream code now.

This FIXME comment was just to double-check that the code is correct. It
looks good to me, so remove the comment.

The window functions rewrite made some changes to pg_dump, to cope with the
catalog changes. cdb_dump_agent is a heavily-modified copy of pg_dump, so
it needs to be changed accordingly.

This is a small batch of commits, to just before the refactoring of the
reloptions machinery. Stopping just before that, because merging that one
requires a bit more work, to also refactor the GPDB parts.

The bulk of this commit is uninteresting churn in the file headers, because
the year in copyright notices was bumped from 2008 to 2009. But one notable
change is the introduction of pg_stat_statements.

The condition was reversed - we should WAL-log the creation, if it's *not*
a temporary relation. This was fixed in the upstream in commit 74ef810c,
which we would merge soon anyway, but there are some complications in GPDB:

In GPDB, this code in smgrcreate() to WAL-log the creation of a new
relation had been removed. Instead, the caller of smgrcreate() calls other
GPDB-specific functions to WAL-log the creation, as an MMXLOG_* record.
But when we merged the relation forks stuff, which moved this code from
smgr.c to storage.c, this code to WAL-log the creation of a new relation
as an SMGR_CREATE record was resurrected. But it didn't actually do
anything because the condition was reversed. Now that we're fixing the
condition, we have to expxlicitly disable it.

With the upcoming WAL replication work, we should WAL log it like in the
upstream. (I haven't tested this with WAL replication enabled, though, so
I'm not sure if we need to do something else, too, to make it actually
work correctly.)

We have lately seen a lot of failures in test cases related to
partitioning, with errors like this:

select tablename, partitionlevel, partitiontablename, partitionname, partitionrank, partitionboundary from pg_partitions where tablename = 'mpp3079a';
ERROR:  cache lookup failed for relation 148532 (ruleutils.c:7172)

The culprit is that that the view passes a relation OID to the
pg_get_partition_rule_def() function, and the function tries to perform a
syscache lookup on the relation (in flatten_reloptions()), but the lookup
fails because the relation was dropped concurrently by another transaction.
This race is possible, because the query runs with an MVCC snapshot, but
the syscache lookups use SnapshotNow.

This commit doesn't eliminate the race completely, but at least it makes it
narrower. A more reliable solution would've been to acquire a lock on the
table, but then that might block, which isn't nice either.

Another solution would've been to modify flatten_reloptions() to return
NULL instead of erroring out, if the lookup fails. That approach is taken
on the other lookups, but I'm reluctant to modify flatten_reloptions()
because it's inherited from upstream. Let's see how well this works in
practice first, before we do more drastic measures.

In the upstream, these keywords are unreserved, and there's no reason they
can't be unreserved in GPDB as well. Take the upstream grammar code to make
that happen. Somehow I missed this in the big window functions rewrite.

In the passing, also remove the useless 'opt_window_order_clause'
production. It's identical to 'opt_sort_clause', and in the upstream, we
use that directly as well.

I bumped into this while looking at the next 8.4 commits we're about to
merge. We're about to bring in pg_stat_statements, and a function used
there has an output parameter called "rows", which doesn't work if it's
a reserved keyword.

A mismerge. Surprisingly it worked despite the duplication. Although I
think there may have been a memory leak, because of the duplicated
MemoryContextSwitchTo call.