In the first cut of the walreplication work, I left out the part that
drops a segfile during AO compaction, because there was no
straightforward native PostgreSQL function to drop just a single segfile.
However, that leads to a massive disk space leak, as the space freed by
VACUUM on an AO table is not reclaimed, until the same segment number is
used again for further insertions, so we clearly must do something about
that.

Instead of dropping the segfile altogether, like we used to, truncate the
file to 0 bytes. Mostly because that's more expedient to do, as we have
the functions to do that and to WAL-log that, whereas code to fully remove
just a single segfile doesn't exist. But if you squint a bit, you could
even call that an optimization: it saves the overhead of recreating the
file again, if you load more data to it later.

Pointed out by Coverity.

As Coverity pointed out, 'pelem' has already been dereferenced in the
function, and hence it can never be NULL here. And indeed it should never
be NULL.

With 'prule', we explicitly checked for NULL earlier in the function.

A PL/JAVA NULL point reference has been reported by coverity scan.
The issue is located in guc.c for a pljava guc option
"pljava_classpath_insecure" and fixed in this commit.

tuple store which just advances current position by N instead of getting
tuple N times.

Author: Max Yang <myang@pivotal.io>

The timezone data in Greenplum are from the base version of
PostgreSQL that the current version of Greenplum is based on.
This cause issues since it means we are years behind on tz
changes that have happened. This pulls in the timezone data
and code from PostgreSQL 10.1 with as few changes to Greenplum
as possible to minimize merge conflicts. The goal is to gain
data rather than features, and for Greenplum for each release
to be able to stay current with the iana tz database as it is
imported into upstream PostgreSQL.

This removes a Greenplum specific test for the Yakutsk timezone
as it was made obsolete by upstream tz commit 1ac038c2c3f25f72.

This is mostly cherry-pick of upstream commit
commit 970a1868
Author: Robert Haas <rhaas@postgresql.org>
Date:   Fri Dec 3 08:44:15 2010 -0500

    Use GUC lexer for recovery.conf parsing.

    This eliminates some crufty, special-purpose code and, as a non-trivial
    side benefit, allows recovery.conf parameters to be unquoted.

    Dimitri Fontaine, with review and cleanup by Alvaro Herrera, Itagaki
    Takahiro, and me.

Originally, we skip probing primary without mirrors, however, we still
use the num_primary_segments to indicate number of requests. This caused
inconsistency when processing the response, since, not all the primary
segments got probe response.

The fix is to only track the actual number of probe requests sent.

Author: Xin Zhang <xzhang@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

- detect primary goes down
- flip the role to m/p/d/n and p/m/u/n (role/prefer/status/mode) in
  gp_segment_configuration
- send promotion message to mirror to promote it

Author: Xin Zhang <xzhang@pivotal.io>
Author: Jacob Champion <pchampion@pivotal.io>
Author: Asim R P <apraveen@pivotal.io>

It set the master_mirroring_administrator_disable variable, but it's not
used for anything. Remove the variable, and all the code to pass that flag
through.

It was now unused.

SetSyncStandbysDefined and friends are now much simpler; there isn't
really anything interesting to test anymore. We already have an
integration test that covers the operation for the whole cluster.

Revert SyncRepUpdateSyncStandbysDefined to its upstream contract: only
checkpointer may call it, and it acquires the SyncRepLock on its own.
If we don't need to ensure that the FTS responder sends back
syncrep=false immediately, the logic is vastly simplified, and we can
remove the assertion on the receiver end.

Also note that we don't need to set the GUC in memory directly; the
SIGHUP will load the new GUC from disk, further simplifying things here.

Author: Jacob Champion <pchampion@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

gp_remove_segment_persistent_entries() function doesn't exist any more.

As a result of removing filespace, we need to pass the datadir instead.
That way, gp_add_segment_primary now matches gp_add_segment_mirror.

Author: Marbin Tan <mtan@pivotal.io>
Author: Jacob Champion <pchampion@pivotal.io>

On further inspection, the code seems perfectly fine.

This reverts commit 0277b584 (which was already partially reverted in
0039844) and commit 4b4643f8. Commit 2b939748 already applied the
logical intent of these commits, as well as some improvements which were
accidentally reverted.

We will not be supporting these utilities in GPDB 6.

References to gpcrondump and gpdbrestore in the gpdb-doc directory have been left
intact, as the documentation will be updated to refer to gpbackup and gprestore
in a separate commit.

Author: Jamie McAtamney <jmcatamney@pivotal.io>

Server side should exclude sub directory even parent directory
wanted.

We don't need the extra 'xid' arguments or the 'frozen' variants of
heap_delete() and heap_update() anymore. They were used by persistent
tables.

Since we have backported streaming replication from PostgreSQL 9.0, we
need to use XLogIsNeeded() rather than XLogArchivingActive(), to determine
if WAL-logging is needed.

This explains the 'CS_switch' TINC test failure.

* Change test cases for gpinistandby -F parameter.
* Fix get standby data directory by gp_segment_configuration instead of pg_filespace_entry
* Fix gpinitstandby to follow old behavior if option -d not given

Author: Xiaoran Wang <xiwang@pivotal.io>
Author: Max Yang <myang@pivotal.io>

A timeline history file would be generated to track timeline change,
and we copy the last wal file as wal file of current timeline. The split
point in history file would take care of correct recovery.

Author: Xiaoran Wang <xiwang@pivotal.io>
Author: Max Yang <myang@pivotal.io>

Make sure that recovery.conf file does not exist after promotion.
The file is renamed to recovery.done, like in upstream.

The previous patch failed, since we could not actually access the
catalog without some more initialization. Pull as much initialization
from the old Pass4 as possible -- some of it is likely not needed, but
this seems to get things working for now.

We had allowed it in GPDB, when we had all the fancy persistent table
and filespace mechanisms to track the created-but-not-yet-committed
tablespace creation. The upstream mechanism is more crude, and can't
handle that gracefully. For example, if you abort the CREATE TABLESPACE
transaction, the directory structure would be left behind. Or worse,
when you DROP TABLESPACE, the underlying directory is deleted immediately,
even if the transaction aborts. Hence, put back the upstream limitation.
That's been acceptable for Postgres, and is surely acceptable for GPDB
as well.

Backport from PostgreSQL 9.0:

commit db64a9d20eeb0d3a52b7fb520a88bc68b73b374b
Author: Bruce Momjian <bruce@momjian.us>
Date:   Tue Jul 20 18:14:25 2010 +0000

    Properly replay CREATE TABLESPACE during crash recovery by deleting
    directory/symlink before creation.

    Report from Tom Lane.

    Backpatch to 9.0.

This change help `FtsTestConnection()` to correctly report segment
state. `probeWalRepPublishUpdate()` updating fts_status if
primary/mirror gets marked as down.

Also, removed all other states and code setting the same as its not
used for anything.

As a followup to the removal of multipass logic in commit 813b817c:

* Make sure that recovery.conf file does not exist after promotion.
  The file is renamed to recovery.done, like in upstream.

* Update gp_segment_configuration after promotion only if we are a
  master's standby.

* We can't actually access the catalog without some more
  initialization. Pull as much initialization from the old Pass4 as
  possible -- some of it is likely not needed, but this seems to get
  things working for now.

Author: Asim R P <apraveen@pivotal.io>
Author: Jacob Champion <pchampion@pivotal.io>