This reverts commit c59e3c8f which
breaks madlib test cases unexpectedly.

This commit makes access to pg_resgroup a little bit faster

see PR 4619

In ResGroupDropFinish() uninitialized memory address can be accessed due
to some reasons:

1. the group pointer is not initialized on segments;
2. the hash table node pointed by group is recycled in removeGroup();

This invalid access can cause crash issue on segments.

Also move some global vars to resgroup.c,
They were put in resgroup-ops-linux.c, which was only compiled and
linked on linux, so on other OS like macos the vars can not be found.

binary_swap_gpdb was input of all resgroup jobs, but as it's not `get` by the resgroup sles job an error will be triggered:

    missing inputs: binary_swap_gpdb

Fixed by marking it as optional.

Bring back the resgroup memory auditor feature:

- 4354d336
- 8ede074c
- 140d4d2e

Memory auditor was a new feature introduced to allow external components
(e.g. pl/container) managed by resource group.  This feature requires a
new gpdb dir to be created in cgroup memory controller, however on 5X
branch unless the users created this new dir manually then the upgrade
from a previous version would fail.

In this commit we provide backward compatibility by checking the release
version:

- on 6X and master branches the memory auditor feature is always enabled
  so the new gpdb dir is mandatory;
- on 5X branch only if the new gpdb dir is created with proper
  permissions the memory auditor feature could be enabled, when it's
  disabled `CREATE RESOURCE GROUP WITH (memory_auditor='cgroup') will fail
  with guide information on how to enable it;

Binary swap tests are also provided to verify backward compatibility in
future releases.  As cgroup need to be configured to enable resgroup we
split the resgroup binary swap tests into two parts:

- resqueue mode only tests which can be triggered in the
  icw_gporca_centos6 pipeline job after the ICW tests, these parts have
  no requirements on cgroup;
- complete resqueue & resgroup modes tests which can be triggered in the
  mpp_resource_group_centos{6,7} pipeline jobs after the resgroup tests,
  these parts need cgroup to be properly configured;

* resgroup: make cgroup memsw.limit_in_bytes optional.
* resgroup: retry proc migration for rmdir to succeed.
* resgroup: add delay in a testcase.
* resgroup: use correct log level in cgroup ops.

backport of https://github.com/greenplum-db/gpdb/pull/4854

* docs: Add guc verify_gpfdists_cert

-added guc definition to list of gucs
-added link to guc from appropriate topics.

PR for 5X_STABLE
Will be ported to MAIN

* docs:  verify_gpfdists_cert guc updates
-add SSL exceptions that are ignored
-other minor edits

* docs: guc verify_gpfdists_cert - fix typos

Executing a query plan containing a large number of slices may slow down
the entire Greenplum cluster: each "n-gang" slice corresponds to a
separate process per segment. An example of such queries is a UNION ALL
atop several complex views. To prevent such a situation, add a GUC
gp_max_slices and refuse to execute plans of which the number of slices
exceed that limit.
Signed-off-by: NJesse Zhang <sbjesse@gmail.com>

This GUC determines whether curl verifies the authenticity of the
gpfdist's certificate

This reverts commit 1697059e.

Due to this commit, full join queries with condition on varchar columns
started failing due to the below error. It is expected that there is a
relabelnode on top of varchar columns while looking up the sort
operator, however because of the said commit we removed the relabelnode.

create table foo(a varchar(30), b varchar(30));
postgres=# select X.a from foo X full join (select a from foo group by 1) Y ON X.a = Y.a;
ERROR:  could not find member 1(1043,1043) of opfamily 1994 (createplan.c:4664)
Will reopen the issue #4175 which brought this patch as we need to rethink the solution.

Previously, gprecoverseg ran a query for an individual segment's mirror state
sequentially against individual segments, which was noticeably slow for large
clusters.  This commit modifies the check to run a single query up front to
get the mirror state for all segments at once and speed things up.

Similar to bin_gpdb_centos6_icw_green, this can be used by downstream
builds that need a GPDB that is passing tests.

See also PR #4399
Authored-by: NDavid Sharp <dsharp@pivotal.io>

The patch 62aba765 from upstream fixed
the CVE-2009-4136 (security vulnerability) with the intent to properly
manage session-local state during execution of an index function by a
database superuser, which in some cases allowed remote authenticated
users to gain privileges via a table with crafted index functions.

Looking into the details of the CVE-2009-4136 and related CVE-2007-6600,
the patch should ideally have limited the scope while we calculate the
stats on the index expressions, where we run functions to evaluate the
expression and could potentially present a security threat.

However, the patch changed the user to table owner before collecting the
sample, due to which even if analyze was run by superuser the sample
couldn't be collected as the table owner did not had sufficient
privileges to access the table. With this commit, we switch back to the
original user while collecting the sample as it does not deal with
indexes, or function call which was the original intention of the patch.

Upstream did not face the privilege issue, as it does block sampling
instead of issuing a query.

Authored-by: NDavid Sharp <dsharp@pivotal.io>
(cherry picked from commit 91bb0d6f)

This is just refactoring. Related to (#4690).
Co-authored-by: NBen Christel <bchristel@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>

This was found when server encoding was WIN1251 but the client
encoding was UTF8. The conversion is valid for these character sets
but the client would receive an "invalid byte sequence for encoding"
error. We fix the issue by calling the correct conversion function. It
was switched up during a postgres backport last year.

Thanks to Heikki for debugging the issue.

In order to more aggressively reduce database age after running this test, we
use VACUUM FREEZE. This should help to minimize the chance that subsequent
tests hit transaction wraparound.

Also cleaned up dita formatting.

-add ENCODING, DEFAULT clauses ADD COLUMN to syntax diagram
-remove restriction "When adding a column to an append-optimized table a DEFAULT clause is required."

PR for 5X_STABLE
Will be ported to MAIN

With this framework, extension developers can set
idle_session_timeout_action_hook and get_idle_session_timeout_hook
to implement actions to take after a session has gone idle for a
specified time period (e.g. killing it).

The code assumes that the (idle gang timeout) < (idle session timeout);
if the reverse happens the session is killed at (idle gang timeout).
Co-authored-by: NDavid Sharp <dsharp@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
Co-authored-by: NBen Christel <bchristel@pivotal.io>

The functionality for gp_vmem_idle_resource_timeout is implemented by
these files. Refactor to make it TDDable and easier to understand.

We moved DisconnectAndDestroyUnusedGangs to cdbgang because it will be a
natural seam for our mocks when we write more unit tests that expect it
to be called. Also, it seems to belong to that file given all of the
functions it calls are already in cdbgang.
Co-authored-by: NBen Christel <bchristel@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>

1) CCP migration from AWS to GOOGLE.

CCP jobs (except for jobs need connection to ddboost and netbackup) now no
longer need external workers, therefore ccp tags for external workers are
removed.

The tfstate backend for AWS and GOOGLE are stored seperatedly on s3 bucket,
`clusters-aws/` for aws and `clusters-google/` for google, set_failed are also
different between the two cloud providers.

2) Separate gpinitsystem from the gen_cluster task

When failures occur in production for gpinitsystem itself, it is important for
a developer to be able to quickly distinguish whether it is a CCP failure, or a
problem with the binaries used to init the GPDB cluster. By separating the
tasks, it is easier to see when gpinit itself has failed.

3) The path to scripts used in CCP has changed

Instead of all of the generic scripts being in `ccp_src/aws/` they are now in a
better location of `ccp_src/scripts/`.

4) Paramater names have changed

platform is now PLATFORM for all references in CCP jobs

5) NVME jobs

Jobs that used NVME in AWS have been migrated to an identical feature for NVME
in GCP but this does include a change to the terraform path specified in the
job.

6) Instance types mapping from Ec2 to GCE

The new paramater name for specifying instance type in GCP jobs is
`instance_type`.  There is not always a 1:1 match for instance types so there
are slight differences in available resources for some jobs.
Signed-off-by: NAlexandra Wang <lewang@pivotal.io>

Signed-off-by: NAlexandra Wang <lewang@pivotal.io>

* docs - add resource group faq

* incorporate edits from david

* docs: autovacuum for template0

* docs: update autovacuum information based on review comments.

* docs: gpfdist-  add note about GPFDIST_WATCHDOG_TIMER env. var. for debugging.

PR for 5X_STABLE
Will be ported to MAIN

* docs: gpfdist - clarified GPFDIST_WATCHDOG_TIMER text.

* docs: gpfdist - add GPFDIST_WATCHDOG_TIMER info to Admin Guide