* update pl/r docs to use CREATE EXTENSION

* update pl/perl, /python, update deprecation notices, add CASCADE info

* Feedback from Chuck

* Update postgresql.org hrefs to 9.4

* Feedback from Lisa

Since Greenplum is now based on PostgreSQL 9.4, update links to the
upstream documentation to refer to the 9.4 docset rather than older
versions. Also fix a few mentions of 8.3(.23) in the text along with
some spacing.
Reviewed-by: NDavid Yozie <dyozie@pivotal.io>

Commit 4e337a33 enabled RECURSIVE
CTEs by default, update the docs to reflect the current state of
the GUC.
Reviewed-by: NLisa Owen <lowen@pivotal.io>
Reviewed-by: NDavid Yozie <dyozie@pivotal.io>

Commit 4e337a33 enabled RECURSIVE
CTEs by default, so let's remove the final instances where we did
override the previous default value.
Reviewed-by: NLisa Owen <lowen@pivotal.io>
Reviewed-by: NDavid Yozie <dyozie@pivotal.io>

The contrib/ directory in Greenplum had over the years had multiple
files removed for various reasons. This tends to create confusion
and pointless merge conflicts as we merge with upstream. This brings
back a lot of files we generally don't really care about, to align
us with upstream. See below for discussions on the various files:

* adminpack is already imported into Greenplum core in order to
  better support pgAdmin3
* earthdistance isn't supported as it relies on upstream features
  not present in Greenplum
* lo is not supported in Greenplum as large objects are disabled
* chkpass and intagg are uninteresting and obsolete modules which
  are being removed from upstream

We found that if gp_segment_configuration is locked, then it will
fail by triggering FTS. We got the stack below
	#2  0x0000000000a6bb29 in ExceptionalCondition at assert.c:66
	#3  0x0000000000aac19a in enable_timeout timeout.c:143
	#4  0x0000000000aacb6c in enable_timeout_after timeout.c:473
	#5  0x00000000008e86ef in ProcSleep at proc.c:1300
	#6  0x00000000008deb70 in WaitOnLock at lock.c:1894
	#7  0x00000000008e019e in LockAcquireExtended at lock.c:1205
	#8  0x00000000008dd2d8 in LockRelationOid at lmgr.c:102
	#9  0x000000000051c928 in heap_open at heapam.c:1083
	#10 0x0000000000b7feaf in getCdbComponentInfo at cdbutil.c:173
	#11 0x0000000000b81365 in cdbcomponent_getCdbComponents at cdbutil.c:606
	#12 0x00000000007603e1 in ftsMain at fts.c:351
	#13 0x0000000000760715 in ftsprobe_start at fts.c:121
	#14 0x00000000004cc7b0 in ServerLoop ()
	#15 0x00000000008769bf in PostmasterMain at postmaster.c:1531
	#16 0x000000000079098b in main ()
So it is that FTS hasn't initialized timeout. Any process that
wants to use timeout must call initilization first. This is the
root cause gpexpand job fails on master pipeline in build 71 and 79.
We added this initialization in FTS and GDD.

GPDB uses libcurl-gnutls.so.4 as default libcurl on ubuntu. gpfdist SSL
connection reports error (56 - Failure when receiving data from the peer)
on handling POST message. We find it shutdown socket directly without sending
close_notify to client. So we call SSL_shutdown() before socket shutdown.

But SSL_accept() will return error with above patch on Centos, because
GPDB curl reuses SSL session ID in the second client hello, but server considers
that session is shutdown, so it won't accept that session ID.
The solution is disabling SSL session ID cache by setting curl option.
Then both Centos and ubuntu work well.

Enable gpfdist SSL test case.

DTX_STATE_ACTIVE_NOT_DISTRIBUTED indicates the transaction is started on QD and
two-phase commit is not required yet, in this state, distributed transaction ID
is also generated. If this transaction is not a two-phase transaction, e.g. implicit
select-only transaction, it will also need to acquire the ProcArrayLock to clear
the distributed transaction ID, and that leads to unnecessary lock contention.

PrepareTempTablespaces is called by almost all callers of
BufFileCreateTemp* and seems like it would be easier from a readability
perspective and as a developer when writing some code that is going to
make temp files, I would prefer not to have to remember to check and set
the value that the user set for temp tablespaces GUC. It seems like that
is an unnecessary chore that is easily forgotten.

For example, for HashAgg, we have our own spilling implementation and
forgot to call PrepareTempTablespaces, so spill files were not getting
created in the temp tablespace specified by the user.

Note that in the workfile_mgr_test, we use SetConfigOption to set the
temp_tablespaces GUC. It would also have worked to redeclare the
variable in the test file and then assign to it. However, SetConfigOption
is used when a user sets a GUC, so it seems more intuitive for the
reader of the test to use this.
Co-authored-by: NAshwin Agrawal <aagrawal@pivotal.io>

concourse/scripts/gppkg_behave.sh, which installs an identical copy of
GPHOME into a new location, existed for exactly one test. Modify that
test to perform its own copy-install.

gpexpand requires centos6, ICW, and CLI jobs to be enabled, but it was
being included in the groups when only a subset of those requirements
was met, so fly refused the pipeline. With this change, centos6/CLI
pipelines should work again.

Int8 for metricsQueryType is big enough, int is not necessary

This refactors the code to remove the separate step for setting the
eol_type in CopyEolStrToType(), which induced a second pointless
pg_strcasecmp() on the passed NEWLINE string. This function did more
originally but upstream merges have made it redundant, and the
function is refers to has never been in the Greenlpum code at all.
Reviewed-by: NAsim R P <apraveen@pivotal.io>

Add the missing code and testcases from upstream to make the ltree
contrib module complete. With this the module builds and tests.

The ignore block in the test case is to avoid a large diff in the
expected file since the test in question fails on the below error
in Greenplum:

  ERROR:  UNIQUE and DISTRIBUTED RANDOMLY are incompatible

Import, and adapt, the upstream testfiles for contrib/btree_gist in
order to make it compile and test green in Greenplum.

The output for these rules have either never existed, or has been
changed over time such that the rule is no longer hitting anything.
Remove to shave hairs off the regress test runtimes.
Reviewed-by: NAshwin Agrawal <aagrawal@pivotal.io>
Reviewed-by: NJimmy Yih <jyih@pivotal.io>

Update sample.gppkg, a workaround to fix gppkg pipeline

Using `yaml.load()` is considered deprecated and unsafe. See the PyYAML
wiki [0] for more details.

[0]: https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation

Currently there are infrastructure issues affecting the maintainability
of these jobs. Until we can address these underlying issues, all CI jobs
related to AIX are being removed.
Co-authored-by: NPeifeng Qiu <pqiu@pivotal.io>
Co-authored-by: NBradford D. Boyle <bboyle@pivotal.io>
Co-authored-by: NKris Macoskey <kmacoskey@pivotal.io>

Skip gprecoverseg tablespace tests until tablespaces successfully work
with pg_rewind.
Authored-by: NKalen Krempely <kkrempely@pivotal.io>

Co-authored-by: NJacob Champion <pchampion@pivotal.io>

Co-authored-by: NJacob Champion <pchampion@pivotal.io>

Co-authored-by: NJacob Champion <pchampion@pivotal.io>

Without synchronous_commit = 'remote_apply' introduced in 9.6, there
is no guarantee that the mirrors have removed their tablespace
directories by the time the DROP TABLESPACE command returns.
We need those directories to no longer be in use by the mirrors
before removing them.

We are using the PL/PGSQL function
checkpoint_and_wait_for_replication_replay() from
src/test/walrep/sql/missing_xlog.sql to achieve the same semantics as
'remote_apply'.
Co-authored-by: NJacob Champion <pchampion@pivotal.io>

Verify tablespaces work for both a incremental and full recovery.
Co-authored-by: NJacob Champion <pchampion@pivotal.io>
Authored-by: NKalen Krempely <kkrempely@pivotal.io>

Co-authored-by: NBhuvnesh Chaudhary <bchaudhary@pivotal.io>
Co-authored-by: NAbhijit Subramanya <asubramanya@pivotal.io>

* gpbackup and gprestore will not run if expand in process

* Add backup step to overview and checklist

* update pl/r docs to use CREATE EXTENSION

* update pl/perl, /python, update deprecation notices, add CASCADE info

* Feedback from Chuck

* Update postgresql.org hrefs to 9.4

* Feedback from Lisa

* docs - address pxf jdbc ddl vs. server config

* misc edits requested by david

This corresponds to ORCA commit "Predicate pushdown in part tables with window func"
It includes ICG changes

Our goal with these pipelines is to find flakes, and the timeouts are getting in the way of that goal.

The answerfile was generated against 5X.  A lot has changed in
numeric.c since then, so update answer file accordingly.

Binary swap tests need to be enabled only on stable branches but not on master.
This commit makes it easy to enable / disable binary swap tests in a pipeline,
especially when gen_pipeline.py is used to generate it.

Compilation with --enable-testutils seems to has been broken long time
as `MaxDynamicLWLock` is no more defined. Given no one complained
seems this is unused config option and code. Hence, instead of fixing
the compilation ripping out this code.

* docs - note for pxf parquet date conversion

* localize to system time first