It was very difficult to verify if interconnect is stucked in resending
phase or if there is udp resending latency within interconnect. To improve
it, this commit record a debug message every Gp_interconnect_debug_retry_interval
times when gp_log_interconnect is set to DEBUG.

This gets rid of the GPDB-specific "QE details" message, that was only sent
once at QE backend startup, to notify the QD about the motion listener port
of the QE backend. Use a standard ParameterStatus message instead, pretending
that there is a GUC called "qe_listener_port". This reduces the difference
between the gp_libpq_fe copy of libpq, and libpq proper. I have a dream that
one day we will start using the standard libpq also for QD-QE communication,
and get rid of the special gp_libpq_fe copy altogether, and this is a small
step in that direction.

In the passing, change the type of Gp_listener_port variable from signed to
unsigned. Gp_listener_port actually holds two values: the TCP and UDP
listener ports, and there is bit-shifting code to store those two 16-bit
port numbers in the single 32-bit integer. But the bit-shifting was a bit
iffy, on a signed integer. Making it unsigned makes it more clear what's
happening.

Resource group memory spill is similar to 'statement_mem' in
resource queue, the difference is memory spill is calculated
according to the memory quota of the resource group.

The related GUCs, variables and functions shared by both resource
queue and resource group are moved to the namespace resource manager.

Also codes of resource queue relating to memory policy are refactored in this commit.
Signed-off-by: NPengzhou Tang <ptang@pivotal.io>
Signed-off-by: NNing Yu <nyu@pivotal.io>

This commit restore TCP interconnect and fix some hang issues.

* restore TCP interconnect code
* Add GUC called gp_interconnect_tcp_listener_backlog for tcp to control the backlog param of listen call
* use memmove instead of memcpy because the memory areas do overlap.
* call checkForCancelFromQD() for TCP interconnect if there are no data for a while, this can avoid QD from getting stuck.
* revert cancelUnfinished related modification in 8d251945, otherwise some queries will get stuck
* move and rename faultinjector "cursor_qe_reader_after_snapshot" to make test cases pass under TCP interconnect.

database.

Signed-off-by Richard Guo <riguo@pivotal.io>
Signed-off-by Gang Xiong <gxiong@pivotal.io>

Works include:
* define structures used by resource group in shared memory;
* insert/remove shared memory object when Create/Drop Resource Group;
* clean up and restore when Create/Drop Resource Group fails;
* implement concurrency slot acquire/release functionality;
* sleep when concurrency slot is not available, and wake up others when
releasing a concurrency slot if necessary;
* handle signals in resource group properly;

Signed-off-by Richard Guo <riguo@pivotal.io>
Signed-off-by Gang Xiong <gxiong@pivotal.io>

gp_resource_manager is reserved for later use to switch resource control stratagy from resource queue to an under-developing resource statagy named resource group. Eg:

gpconfig -c gp_resource_manager -v 'group'
gpconfig -c  gp_resource_manager -v 'queue'

To make it work, a restart of cluster is needed.

The gp_enable_alter_table_inherit_cols GUC was used to allow a list
of columns to override the attribute discovery for inheritance in
ALTER TABLE INHERIT. According to code comments the only consumer
of this was gpmigrator, but no callsite remains and no support in
pg_dumpall remains either. Remove the leftovers of this to avoid a
potential footgun and get us closer to upstream code.

The gp_external_grant_privileges GUC was needed before 4.0 to let
non superusers create external tables for gphdfs and http protocols.
This GUC was however deprecated during the 4.3 cycle so remove all
traces of it. The utility of the GUC was replaced in 4.0 when rights
management for external tables was implemented with the normal GRANT
REVOKE framework so this has been dead code for quite some time.

Remove GUC, code which handles it, all references to it from the
documentation and a release notes entry.

The gp_eager_hashtable_release GUC was deprecated in version 4.2
in 2011 when the generic eager free framework was implemented. The
leftover gp_eager_hashtable_release was asserted to be true and
never intended to be turned off. The same body of work deprecated
the max_work_mem setting which was bounding the work_mem setting.
While not technically tied to eager hashtable release, remove as
well since it's deprecated, undocumented and not terribly useful.

Relevant commit in closed source repo is 88986b7d

The gp_hashagg_compress_spill_files GUC was deprecated in 2010 when
it was replaced by gp_workfile_compress_algorithm. The leftovers
haven't done anything for quite some time so remove GUC.

Relevant commit in closed source repo is c1ce9f03

Signed-off-by Gang Xiong <gxiong@pivotal.io>

It hasn't done anything since 2010. If I'm reading the commit log
correctly, it was added and deprecated only a few months apart, and
probably hasn't done anything in any released version.

And other misc cleeanup.

QE reader leverages SharedLocalSnapshot to perform visibility checks. QE writer
is responsible to keep the SharedLocalSnapshot up to date. Before this fix,
SharedLocalSnapshot was only updated by writer while acquiring the snapshot. But
if transaction id is assigned to subtransaction after it has taken the snapshot,
it was not reflected. Due to this when QE reader called
TransactionIdIsCurrentTransactionId, it may get sometimes false based on timings
for subtransaction ids used by QE writer to insert/update tuples. Hence to fix
the situation, SharedLocalSnapshot is now updated when assigning transaction id
and deregistered if subtransaction aborts.

Also, adding faultinjector to suspend cursor QE reader instead of guc/sleep used
in past. Moving cursor tests from bugbuster to ICG and adding deterministic test
to exercise the behavior.

Fixes #1276, reported by @pengzhout

Ensure that that the header guards match the actual name of the
file.

Remove unused fields from past version control systems and ensure
that all filenames in the comments match the actual name of the file.
Also fix some spelling and references.

The previous commits have removed all usage of CaQL. It's no longer
needed.

The feature was disabled, by removing it from guc.c, only months after
it was introduced, back in 2009. I'm not sure if it ever made it into a
public release. We have no plans to resurrect it, and it has surely
bitrotted so badly that even if we did, it would be better to start from
scratch.

Codes of ic_tcp and ic_udp are not maintained any more, ic_udpifc is
the only interconnect type for now. We keep the gp_interconnect_type
guc for backward compatibility.