CI reported some read failures just after the write action, it's
probably because the write action is not flushed yet by AWS.

Sleep and retry if AWS returns "NoSuchKey" error. Update the workflow
but not the test cases because users might have the same issue.
Co-authored-by: NPeifeng Qiu <pqiu@pivotal.io>

Includes ICG changes for ORCA commit:
'Convert FULL OUTER JOIN to LEFT JOIN when possible'

Authored-by: NChris Hajas <chajas@pivotal.io>

Also add a few more tests for unions to ICG.

(cherry picked from commit 96f34a8c)

Because users may find it confusing when gpconfig prints '-' or 'None'
when no GUC value is found in the file, this commit updates gpconfig to
output a clearer message.

Apply the out-of-band messaging to failure reports as well, and fix up
the output of --file-compare. Add tests for each modified
implementation.
Co-Authored-By: NJamie McAtamney <jmcatamney@pivotal.io>
Co-Authored-By: NMark Sliva <msliva@pivotal.io>
Co-Authored-By: NShoaib Lari <slari@pivotal.io>
(cherry picked from commit b8bd3577)

The test should wait for the transactions to be in the right state
before promoting standby.  This commit adds a wait step to ensure just
that.  One of the ICW jobs in CI failed because the test promoted the
standby before the transactions were preprared on master.  This should
no longer happen now.

Transactions that are in the middle of two phase commit are suspended on
master.  Standby is promoted while they are suspended.  Based on what XLOG
records are emitted by master, the standby is expected to perform DTM recovery
and complete the transactions upon promotion.

This feature enables tests to run SQL on standby after it is promoted.  Use
"-1S: <sql>" to run <sql> statement on standby.  It is assumed that the standby
is already promoted.

Skip gprecoverseg tablespace tests until tablespaces successfully work
with pg_rewind.
Co-authored-by: NMark Sliva <msliva@pivotal.io>
(cherry picked from commit aae20c86)

(cherry picked from commit bb532dec)

Co-authored-by: NJacob Champion <pchampion@pivotal.io>
(cherry picked from commit b81ee6cd)

Create the tablespace directory on all segments to support
running multi-host tablespace tests.
Co-authored-by: NJacob Champion <pchampion@pivotal.io>
(cherry picked from commit 83f48aac)

Remove unused behave step:
"the user waits for "{process_name}" to finish running"

(cherry picked from commit 3a3a4b70036925fb0ef11354b76fb20461afe931)

Remove behave step "the database is killed on hosts mdw,sdw1,sdw2" in
favor of "the database is not running. This shuts down the databse more
cleanly, and avoids any potential race conditons.

(cherry picked from commit 8079bd2be75f0624212d7c4907ea16e8c5a5d199)

Remove "user kills a primary postmaster process" in favor of the more
generic equivalent step "user stops all {segment_type} processes" which
can take either a primary or mirror.

For clarity and accuracy rename "user kills all {segment_type}
processes" to "user stops all {segment_type} processes".

(cherry picked from commit 8eb9dffb726a9218166af9ba06a91efb7c492826)

Cleanly shutdown segments using pg_ctl to avoid potential race
conditions by grep'ing for the pid via ps. This is a much simplier
approach to help with maintainability and extensibility.

(cherry picked from commit bd9bbaf837be3d4f7334822d4d14469702725c55)

Previously we error out in gp_replica_check test if the cluster is not in
synced, which is usually due to bugs revealed by previous tests, instead of
gp_replica_check itself.

Print more detailed message so that people are not confused with the status and
this failure reason of the gp_replica_check test.

* Updates to GUCs from Cyrille's reviews

* Revise description of temp_buffers to match 6.0 behavior

* Fix a couple typos

* For effective_cache_size GUC, show default in blocks and size.

DTX_STATE_ACTIVE_NOT_DISTRIBUTED indicates the transaction is started on QD and
two-phase commit is not required yet, in this state, distributed transaction ID
is also generated. If this transaction is not a two-phase transaction, e.g. implicit
select-only transaction, it will also need to acquire the ProcArrayLock to clear
the distributed transaction ID, and that leads to unnecessary lock contention.

We found that if gp_segment_configuration is locked, then it will
fail by triggering FTS. We got the stack below
	#2  0x0000000000a6bb29 in ExceptionalCondition at assert.c:66
	#3  0x0000000000aac19a in enable_timeout timeout.c:143
	#4  0x0000000000aacb6c in enable_timeout_after timeout.c:473
	#5  0x00000000008e86ef in ProcSleep at proc.c:1300
	#6  0x00000000008deb70 in WaitOnLock at lock.c:1894
	#7  0x00000000008e019e in LockAcquireExtended at lock.c:1205
	#8  0x00000000008dd2d8 in LockRelationOid at lmgr.c:102
	#9  0x000000000051c928 in heap_open at heapam.c:1083
	#10 0x0000000000b7feaf in getCdbComponentInfo at cdbutil.c:173
	#11 0x0000000000b81365 in cdbcomponent_getCdbComponents at cdbutil.c:606
	#12 0x00000000007603e1 in ftsMain at fts.c:351
	#13 0x0000000000760715 in ftsprobe_start at fts.c:121
	#14 0x00000000004cc7b0 in ServerLoop ()
	#15 0x00000000008769bf in PostmasterMain at postmaster.c:1531
	#16 0x000000000079098b in main ()
So it is that FTS hasn't initialized timeout. Any process that
wants to use timeout must call initilization first. This is the
root cause gpexpand job fails on master pipeline in build 71 and 79.
We added this initialization in FTS and GDD.

* Docs - update pg_class and pg_index catalog table references

* dyozie review comments

concourse/scripts/gppkg_behave.sh, which installs an identical copy of
GPHOME into a new location, existed for exactly one test. Modify that
test to perform its own copy-install.

(cherry picked from commit e6b40a71)

gpexpand requires centos6, ICW, and CLI jobs to be enabled, but it was
being included in the groups when only a subset of those requirements
was met, so fly refused the pipeline. With this change, centos6/CLI
pipelines should work again.

(cherry picked from commit 52a21767)

  This is different from the gpdb-master,
  The gpdb-master pipeline save the RC artifacts
  to the server/published/master/
Authored-by: NBob Bao <bbao@pivotal.io>

* Add a list of SQL keywords to the ref guide

* Fix error from review

* Update from review comment

Story: https://www.pivotaltracker.com/story/show/164917628

   The gp-integration-testing pipeline needs the gp-clients RC
   package as a input, and then create the clients RPM base on it.
   The custom installer patch also need it to build client bin installer.
Co-authored-by: NXiaoran Wang <xiwang@pivotal.io>
Co-authored-by: NShaoqi Bai <sbai@pivotal.io>
Co-authored-by: NBob Bao <bbao@pivotal.io>

ORCA Explain plans will now contain:
`Optimizer: Pivotal Optimizer (GPORCA) version 3.35.0`

instead of:
`Optimizer: PQO version 3.35.0`

Authored-by: Chris Hajas chajas@pivotal.io

Commit 4e337a33 enabled RECURSIVE
CTEs by default, update the docs to reflect the current state of
the GUC.
Reviewed-by: NLisa Owen <lowen@pivotal.io>
Reviewed-by: NDavid Yozie <dyozie@pivotal.io>

* Update pl/java docs for JVM version, shared library loc, CREATE EXTENSION command, .sql file deprecation

* Add default library location for JDK 8 and JDK 11

* format testdb; relocate uninstall topic

Fix two spotted occurrences of "Greemplum"

Reviewed-by: NDavid Yozie <dyozie@pivotal.io>