The code to look up the hash functions for a Reshuffle plan used
get_opfamily_proc() instead of the more versatile
cdb_hashproc_in_family() function, which is used in other similar places
where we need to look up the hash functions for a distribution key. Like
in makeCdbHashForRelation(). That lead to errors if the datatype didn't
have a hash function defined directly for the datatype, but only via a
binary-coercible cast. Domain and enum types are such cases, for example.

Fixes https://github.com/greenplum-db/gpdb/issues/6901Reviewed-by: NDavid Kimura <dkimura@pivotal.io>
Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

Co-authored-by: NBen Christel <bchristel@pivotal.io>
Co-authored-by: NSambitesh Dash <sdash@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
Co-authored-by: NDavid Sharp <dsharp@pivotal.io>

GPDB 5 and 6 only require xerces-c for gporca. We were originally
pulling it from Ivy during the build process, but it is now bundled with
gporca.
Co-authored-by: NBen Christel <bchristel@pivotal.io>
Co-authored-by: NSambitesh Dash <sdash@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
Co-authored-by: NDavid Sharp <dsharp@pivotal.io>

This reverts commit 1cf5e595.

This reverts commit 07d9a5f8.

* docs - add GUC gp_enable_global_deadlock_detector

Add Note text to DELETE, LOCK, UPDATE about GDD.
Also, update lock descriptions in dml.xml to point to GDD section.

* docs - review updates for gp_enable_global_deadlock_detector GUC

When a primary segment crashes, there is a possibility of the
postmaster.pid being left behind. If another process starts actively
using that pid, pg_rewind will failed when it tries to start the
failed segment in single-user mode (crash recovery to ensure clean
shutdown). It should be safe to remove the file before pg_rewind
during gprecoverseg since gprecoverseg ensures all failed segments are
shut down (pg_ctl stop or via forceful termination).

Also, revert some gprecoverseg full calls in gprecoverseg.feature
Behave test to be back to gprecoverseg incremental. They were marked
with WALREP_FIXME last year waiting for pg_rewind which we now have.
Co-authored-by: NJimmy Yih <jyih@pivotal.io>

Issue is a hardcoded port in the default `net.ipv4.ip_local_port_range`
of 32768 - 60999. It is possible that another processes is using the
port, in which case the test will fail.  This commit changes to a port
outside that range.  Alternatively, We could dynamically fetch free
ports, but that adds extra unnecessary complexity.
Co-authored-by: NAlexandra Wang <lewang@pivotal.io>

This reverts commit 4f686127 as it
has been causing strange failures in the main pipeline.

Commits 41c919ce, 0ad14f38 and 08d7ca18 removed these columns, but
forgot to update the docs accordingly.

Commit ad6b920f accidentally removed
the headerguards due to them being namedd GPHDFS, even though they
aren't GPHDFS related. This caused a lot of redefenition warnings at
build time, which are fixed by this.

In Greenplum 5.X the recursive CTE feature was hidden behind a GUC as
it wasn't deemed of production quality just yet. Commit 20152cbf
removed that GUC in order to make stabilization work easier, there are
still enough rough edges to not consider recursive CTE a feature which
is on by default. This brings back the GUC using the same name in order
to be backwards compatible even though "prototype" is a bit misleading
as this point.

Also adds a test and tidies up a few comments in surrounding code.

Discussion: https://groups.google.com/a/greenplum.org/forum/#!topic/gpdb-dev/Lt6l3wDyxTYCo-authored-by: NGeorgios Kokolatos <gkokolatos@pivotal.io>

* docs - remove/replace references to gphdfs

* misc edits

* stronger statement (note) about pxf replacing gphdfs

* stronger statement

* edit to note requested by david

* haved -> have... doh

* docs - upgrade orafce to 3.7

* review comments

* Updates from review comments

* Notes about orafce upgrade when upgrading gpdb

It's not used for anything.

If gpexpand needed these for something, then it should also take
'distclasses' into account. But the current implementation of gpexpand
doesn't need them. If a user is interested in the distribution policy of
a table, he can connect to the correct database and use "\d table".

The names are not configurable, the variables were just used as constants.
IMHO using the constants just made the code more difficult to read, so
better to use the names directly in the SQL commands. Besides, there were
already a few queries where we didn't use the constants.

gphdfs is being deprecated for GPDB 6. This commit removes the
following:
- Remove gphdfs from pipelines
- Remove gphdfs from gpcontrib
- Remove gphdfs from Makefile
- Remove tests using gphdfs
- Remove GUCs, code, and comments used for gphdfs
- Remove gphdfs from GPDB docs
- Remove avro and parquet formats

We are renaming the docker images we use with concourse to have the gpdb
version in the actual image name rather than in the tag. The contents of
the images will be the same as before.
Authored-by: NKaren Huddleston <khuddleston@pivotal.io>

We left the comment explaining the difference between GPDB
and upstream, as it will be useful for future merge work.

pg_event_trigger_dropped_objects should observe events run
on the QD when tables are dropped and have the same behavior
as upstream.

There's a possible race-condition because the query no longer
joins, so there's a gap between the lookup of dropped objects
and the query for undroppable objects, but because theres no
other queries using this table or function, it is not a concern.
Co-authored-by: NGeorgios Kokolatos <gkokolatos@pivotal.io>

"primary" and "mirror" are clearer than "peer" and "segment", which were
holdovers from a previous implementation. Also switch the order that we
pass these to _add_replication_info(), from (mirror, primary) to
(primary, mirror).
Co-authored-by: NJacob Champion <pchampion@pivotal.io>
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>

To make it obvious when a basebackup is in progress (such as with
`gprecoverseg -F`), if any primary's WAL senders are actively backing up
logs to a receiver, `gpstate -s` will now display 'Copying files from
primary' for that primary's Mirror Status. This is true even if a mirror
is currently synced and streaming.
Co-authored-by: NAdam Berlin <aberlin@pivotal.io>

...so that we can log information for a segment pair all at once without
too much trouble.
Co-authored-by: NAdam Berlin <aberlin@pivotal.io>

Co-authored-by: NKaren Huddleston <khuddleston@pivotal.io>
Co-authored-by: NNandish Jayaram <njayaram@pivotal.io>

We are moving away from using Ivy for managing enterprise GPDB
dependencies. Because ccache is no longer used by anything in the
gpdb_master CI, it is safe to remove.
Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>
Co-authored-by: NBradford Boyle <bboyle@pivotal.io>

As a first step to eventually removing Ivy as a dependency for the
enterprise build of GPDB, remove R from the list of dependencies
downloaded during make sync_tools. This version of R is not used, so no
need to modify anything else in the build process.

Also remove obsolete references to $R_HOME in releng.mk
Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>
Co-authored-by: NBradford Boyle <bboyle@pivotal.io>

We have a high level goal to remove all ivy dependencies for gpdb6.
kfw and pygresql were the only dependencies in this tarball. We are
dropping both from ivy so we no longer need to pull the tarball here.
Co-authored-by: NKaren Huddleston <khuddleston@pivotal.io>
Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>

All libraries and programs from the third-party extensions tarball are
now provided by the operating system, so we no longer need to pull this
artifact from ivy.

Dependencies that were dropped:
 - bzip2
 - libedit
 - libevent
 - libxml2
 - libyaml
 - liblber (left behind in the tarball by accident)
Co-authored-by: NKaren Huddleston <khuddleston@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>

Remove libyaml and liblber libraries from Makefile for centos

We are not providing these libraries from Ivy in rhel6_x86_64 anymore so
they will not be in our ext directory.
Co-authored-by: NKaren Huddleston <khuddleston@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>

This reverts commit 39dfead3
and reapplies commit 39dfead3.

The original commit was reverted due to a test failure in gpcloud, but
the failing test has been disabled.
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
Co-authored-by: NKaren Huddleston <khuddleston@pivotal.io>

This reverts commit 3b04c467.
and reapplies commit 577b09ce.

The original commit was reverted due to a test failure in gpcloud, but
the failing test has been disabled.
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
Co-authored-by: NKaren Huddleston <khuddleston@pivotal.io>

* docs - misc pxf edits

* add pxf to loading/unloading page and subnav

* edit requested by lav