The timezone data in Greenplum are from the base version of
PostgreSQL that the current version of Greenplum is based on.
This cause issues since it means we are years behind on tz
changes that have happened. This pulls in the timezone data
and code from PostgreSQL 10.1 with as few changes to Greenplum
as possible to minimize merge conflicts. The goal is to gain
data rather than features, and for Greenplum for each release
to be able to stay current with the iana tz database as it is
imported into upstream PostgreSQL.

This removes a Greenplum specific test for the Yakutsk timezone
as it was made obsolete by upstream tz commit 1ac038c2c3f25f72.

We can't run gp_replica_check on btree, gin and gist as they are
currently not guaranteeing physical equivalence, only logical.
When we've merged 9.4, we can however start turning this on again
as the pages are then holding the physical equivalence property.

pg_ctl and gpstart appear to be running simultaneously, because we're
not waiting for pg_ctl to finish before moving to the next call.
(subprocess.Popen() doesn't return the process's return code -- it
returns a Popen instance.)

cs_walrep_2 in the master pipeline is currently failing roughly 50% of
the time due to a crash during test_negative. Let's see if this helps at
all.

The temporary segwalrep pipeline can be removed now.

Author: Xin Zhang <xzhang@pivotal.io>
Author: Jimmy Yih <jyih@pivotal.io>

This is to pass isolation2 max concurrency tests.

Author: Xin Zhang <xzhang@pivotal.io>
Author: Jimmy Yih <jyih@pivotal.io>

Author: Xin Zhang <xzhang@pivotal.io>
Author: Jimmy Yih <jyih@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

dbid is not consistent (like content 0 mirror will always have dbid 5), as
segments get created in parallel and get assigned those based on order of
execution. Hence better convention to use is content-id, to specify which
segment to connect. Framework currently connects to primary of specified
content-id.

If later needed, maybe when we enable hot-standby feature can extend further to
start supporting connecting to specified content-ids mirror as well. Not
something needed as of now.

Author: Xin Zhang <xzhang@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

This is mostly cherry-pick of upstream commit
commit 970a1868
Author: Robert Haas <rhaas@postgresql.org>
Date:   Fri Dec 3 08:44:15 2010 -0500

    Use GUC lexer for recovery.conf parsing.

    This eliminates some crufty, special-purpose code and, as a non-trivial
    side benefit, allows recovery.conf parameters to be unquoted.

    Dimitri Fontaine, with review and cleanup by Alvaro Herrera, Itagaki
    Takahiro, and me.

The test uses \retcode to ignore the output of the external script, but
remember the return code in the expected result.

It's very useful for this scenario to ensure, we don't care for the
details of the recoveryfull operation, but we do care this operation
complete successfully.

The test stop the primary and trigger failover to its mirror, then fail
back to the original segment configuration. At the end of the test, the
gp_segment_configuration is the same as the beginning of the test.

Author: Xin Zhang <xzhang@pivotal.io>
Author: Jacob Champion <pchampion@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

Originally, we don't wait for mirror in sync after recoverfull, and the
final state of the cluster would be non-deterministic. Since, we are
doing the full recovery here, hence the final state the mirrors should
always be in-sync with their primaries.

Author: Xn Zhang <xzhang@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

`rebuild` is different from `recoverfull`. It will create new directory
for the new mirror with new port for given content id.

`recoverfull` on the other hand will just recover all the failed mirrors
using existing information in gp_segment_configuration.

The both use basebackup to build the mirror.

We reduce port increment to avoid over the port limit after multiple
rebuild.

Author: Xin Zhang <xzhang@pivotal.io>
Author: Asim R P <apraveen@pivotal.io>

Ignore segments not marked as up for clusterstart operation.

Author: Xin Zhang <xzhang@pivotal.io>
Author: Asim R P <apraveen@pivotal.io>

Author: Xin Zhang <xzhang@pivotal.io>
Author: Asim R P <apraveen@pivotal.io>

Author: Jacob Champion <pchampion@pivotal.io>
Author: Asim R P <apraveen@pivotal.io>

Originally, we skip probing primary without mirrors, however, we still
use the num_primary_segments to indicate number of requests. This caused
inconsistency when processing the response, since, not all the primary
segments got probe response.

The fix is to only track the actual number of probe requests sent.

Author: Xin Zhang <xzhang@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

- detect primary goes down
- flip the role to m/p/d/n and p/m/u/n (role/prefer/status/mode) in
  gp_segment_configuration
- send promotion message to mirror to promote it

Author: Xin Zhang <xzhang@pivotal.io>
Author: Jacob Champion <pchampion@pivotal.io>
Author: Asim R P <apraveen@pivotal.io>

Author: Xin Zhang <xzhang@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

In the past, the utility mode will not be retried, and caused wrong
behavior when trying to connect to given segment under utility mode.

In this refactoring, both utility mode and normal mode will go through
connectdb() that has the retry logic.

Author: Xin Zhang <xzhang@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

We don't want to diff the output from `segwalrep.py recoverfull`, but we
do want to know if the command has succeeded or not. Add a new execution
mode, invoked via

    !\retcode EXECUTABLE [OPTIONS ...]

which will bracket the program's stdout/stderr with begin/end_ignore,
and also print out an exit code (which is not ignored).

Author: Jacob Champion <pchampion@pivotal.io>
Author: Xin Zhang <xzhang@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

If number of segments is not set to a value > 0, the assertion in
getgpsegmentCount() trips.  Executor calls this function from many
places.  Without this change, executing "select * from
gp_stat_replication;" trips the assertion on a promoted mirror because
number of contents is set to 0.

Author: Asim R P <apraveen@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

There is no reason to believe that these bugs could reappear, in the same
form.

It set the master_mirroring_administrator_disable variable, but it's not
used for anything. Remove the variable, and all the code to pass that flag
through.

It was now unused.

There is a race condition in this test, where our disabling of the FTS
probe process doesn't actually have to happen immediately (it takes a
little while for the GUC to propagate). This may be the cause of some
test failures we see where the primary is falling out of sync with the
mirror before we want it to.

Try a basic hack to see if it helps anything; the real fix will be to
move to faultinjection, but that's another story.

Author: Ashwin Agrawal <aagrawal@pivotal.io>
Author: Jacob Champion <pchampion@pivotal.io>

SetSyncStandbysDefined and friends are now much simpler; there isn't
really anything interesting to test anymore. We already have an
integration test that covers the operation for the whole cluster.

Revert SyncRepUpdateSyncStandbysDefined to its upstream contract: only
checkpointer may call it, and it acquires the SyncRepLock on its own.
If we don't need to ensure that the FTS responder sends back
syncrep=false immediately, the logic is vastly simplified, and we can
remove the assertion on the receiver end.

Also note that we don't need to set the GUC in memory directly; the
SIGHUP will load the new GUC from disk, further simplifying things here.

Author: Jacob Champion <pchampion@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

synchronous_standby_names is set on the first FTS probe after starting
the cluster, and it's written to disk, but it doesn't actually get
persisted in memory on all cluster processes. The backend that "sets"
the value exits without SIGHUP'ing the cluster, and it therefore
"resets" to the empty string. If you're lucky enough to restart the
cluster after this point, everything works as expected, but until then
the cluster is in a strange state.

This test (which currently fails) verifies that the
synchronous_standby_names GUC is actually set appropriately on the
segment.

The syncing should be done automatically by the mirror, when the mirror
connects to the primary.

Author: Marbin Tan <mtan@pivotal.io>
Author: Shoaib Lari <slari@pivotal.io>

gp_segment_configuration should be updated through
gp_add_segment_primary and gp_add_segment_mirror

- Add a fix for our status validation check.
We were checking an incorrect of elements in a tuple, it was always
return false if you only had primaries.
Make it such that the mirrors will always be assumed down when
validating when you only have primaries for the cluster.

Author: Marbin Tan <mtan@pivotal.io>

The column is called 'datadir', not 'directory'.

The vacuum was causing trouble, see github issue #4298. And it seems pretty
pointless or unrelated to gpexpand, anyway.

This reverts commit 4efea6f73ef1b828e7ca435a9a62790f787ed861. I found
the culprit, and don't need that debugging information anymore.