Which makes no sense, and will fail while creating the schema, whose
name starts with reserved prefix "pg".

The current testing framework could not hold an extra connection for
testing this.

adding reference topic for maintenance_work_mem, which is mentioned elsewhere in the docs and in recent postgresql 8.4 merges (#4422)

Three-stage aggregate is an optimization to parallelize DISTINCT
aggregate. Except for trivial cases (e.g. there is only one Aggref,
or every Aggref has exactly the same aggfilter),
it seems impossible to apply this class of optimizations to a
query with SELECT aggfn(DISTINCT a) FILTER (WHERE ...)

Co-authored-by: NDavid Kimura <dkimura@pivotal.io>

In concurrent scenarios, globalxmin can be lower than distributed
oldest xmin. This can happen as ProcArrayLock is released before
calling DistributedLog_AdvanceOldestXmin() in
GetSnapshotData(). Earlier it used to ERROR for this case. Now instead
it treats safe to return distributed oldest xmin even if its higher
than current process globalXmin, as some other process already
consulted distributed log to bump the distributed
oldestXmin. Condition which must be satisfied is globalXmin for this
process remains lower or equal to TransactionXmin.
Co-authored-by: NDavid Kimura <dkimura@pivotal.io>

-- only output verbose debian build logging upon failure

[#156761964]
Co-authored-by: NLarry Hamel <lhamel@pivotal.io>
Co-authored-by: NXin Zhang <xzhang@pivotal.io>

see PR 4619

Dispatcher has DISPATCH_WAIT_TIMEOUT_MSEC (current value is 2000) as poll
timeout. It waited for 30 iterations of poll to timeout before checking the
segment status. And then initiated fts probe before checking the segment
status. As a result it took ~minute for query to fail in case of segment
failures.

This commit updates to check segment status on every poll timeout. It also
leverages fts version to optimize whether to check segments. It avoids
performing fts probe, instead it relies on fts to be called on regular
intervals and provide cached results.

With this change test time for twophase_tolerance_with_mirror_promotion was cut
down by ~2 minutes.
Co-authored-by: NAshwin Agrawal <aagrawal@pivotal.io>

The goal of these tests is to validate that existing transactions on the
primary are not hung when the mirror gets promoted. In order to validate
this, mirror promotion is triggered at the following two-phase commit
points:
  1) if the transaction hasn't prepared, then it should be aborted
  2) if the transaction is already prepared, then it should complete commit
     on the mirror
  3) if the transaction has committed on the primary, but not acknowledged
     to the master then it should complete the commit on the mirror
Co-authored-by: NAshwin Agrawal <aagrawal@pivotal.io>

Sometimes test need to use this fault to suspend and then stop the database. If
this fault is inside the HOLD_INTERRUPTS (which is where its before this
change), CHECK_FOR_INTERRUPTS doesn't do anything. Hence stopping of segment
takes super long time as kill -9 needs to be enforced by gpstop after timeout.
Co-authored-by: NDavid Kimura <dkimura@pivotal.io>

In ResGroupDropFinish() uninitialized memory address can be accessed due
to some reasons:

1. the group pointer is not initialized on segments;
2. the hash table node pointed by group is recycled in removeGroup();

This invalid access can cause crash issue on segments.

Also move some global vars to resgroup.c,
They were put in resgroup-ops-linux.c, which was only compiled and
linked on linux, so on other OS like macos the vars can not be found.

binary_swap_gpdb was input of all resgroup jobs, but as it's not `get` by the resgroup sles job an error will be triggered:

    missing inputs: binary_swap_gpdb

Fixed by marking it as optional.

User query can use global shared memory across resource group when available(Same as the PR 4843, just made the test cases stable) (#4866)

1) Global shared memory will be used if the query has run out of
	the group shared memory.
2) The limit of memory_spill_ratio changes to [0, INT_MAX], because
	global shared memory can be allocated, 100% limitation is not
	make sense.
3) Using Atomic Operation & "Compare And Save" instead of lock to
	get high performance.
4) Modify the test cases according to the new rules.

Cgroup mount point is detected during startup, but due to a bug the
mount point was used even if cgroup is not mounted at all.

Fixed by correcting the checking logic.

Memory auditor was a new feature introduced to allow external components
(e.g. pl/container) managed by resource group.  This feature requires a
new gpdb dir to be created in cgroup memory controller, however on 5X
branch unless the users created this new dir manually then the upgrade
from a previous version would fail.

In this commit we provide backward compatibility by checking the release
version:

- on 6X and master branches the memory auditor feature is always enabled
  so the new gpdb dir is mandatory;
- on 5X branch only if the new gpdb dir is created with proper
  permissions the memory auditor feature could be enabled, when it's
  disabled `CREATE RESOURCE GROUP WITH (memory_auditor='cgroup') will fail
  with guide information on how to enable it;

Binary swap tests are also provided to verify backward compatibility in
future releases.  As cgroup need to be configured to enable resgroup we
split the resgroup binary swap tests into two parts:

- resqueue mode only tests which can be triggered in the
  icw_gporca_centos6 pipeline job after the ICW tests, these parts have
  no requirements on cgroup;
- complete resqueue & resgroup modes tests which can be triggered in the
  mpp_resource_group_centos{6,7} pipeline jobs after the resgroup tests,
  these parts need cgroup to be properly configured;

This commit includes the stainherit columns in minirepro output recently
added.

Recent postgres merge introduced a new column `stainherit` to
`pg_statistic` table. When `gpsd` dumps the pg_statistic not only a
column is missing but also all the other columns are shifted and saved
in wrong column index. This causes failure on reloading the gpsd.

This commit fixes the issue and adds the missing column.

We are developing on a machine with flex 2.6.4 and we couldn't build.
This was fixed in 72b1e3a2 in upstream (9.6 at the time) in 2016, but
wasn't backported to 9.1 or older.

Original commit message:
> Now that we know about the %top{} trick, we can revert to building flex
> lexers as separate .o files.  This is worth doing for a couple of reasons
> besides sheer cleanliness.  We can narrow the scope of the -Wno-error flag
> that's forced on scan.c.  Also, since these grammar and lexer files are
> so large, splitting them into separate build targets should have some
> advantages in build speed, particularly in parallel or ccache'd builds.
>
> We have quite a few other .l files that could be changed likewise, but the
> above arguments don't apply to them, so the benefit of fixing them seems
> pretty minimal.  Leave the rest for some other day.

(cherry picked from commit 72b1e3a2)

Fixes greenplum-db/gpdb#4863
Co-authored-by: NJesse Zhang <sbjesse@gmail.com>
Co-authored-by: NAsim R P <apraveen@pivotal.io>

Revert "User query can use global shared memory across resource group when available(#4843)" (#4862)

This reverts commit a5b3831d.

1) Global shared memory will be used if the query has run out of
	the group shared memory.
2) The limit of memory_spill_ratio changes to [0, INT_MAX], because
	global shared memory can be allocated, 100% limitation is not
	make sense.
3) Using Atomic Operation & "Compare And Save" instead of lock to
	get high performance.
4) Modify the test cases according to the new rules.

'distribution_policy' test do constraints check on randomly-distributed
tables, however, attrnums = null in gp_distribution_policy is no longer
effective to identify a randomly distributed table after we involved new
replicated distributed policy, so add more filter to make things right.

Large objects are currently not supported in Greenplum. Rather than deceive the
user with a non-functional large object api, we disable them for now.

We disable the large object tests in privileges regress test by using ignore
blocks instead of commenting them out or deleting them to reduce merge
conflicts in future postgres merges.
Co-authored-by: NJimmy Yih <jyih@pivotal.io>

* docs: Add guc verify_gpfdists_cert

-added guc definition to list of gucs
-added link to guc from appropriate topics.

PR for 5X_STABLE
Will be ported to MAIN

* docs:  verify_gpfdists_cert guc updates
-add SSL exceptions that are ignored
-other minor edits

* docs: guc verify_gpfdists_cert - fix typos

When the blocksize is 2MB, the function AppendOnlyStorage_GetUsableBlockSize
would give out the wrong usable block size. The expected result is 2MB. But the
return value of the function call would give out (2M -4). This is because the
macro AOSmallContentHeader_MaxLength is defined as (2M -1). After rounding down
to 4 byte aligned, the result is (2M - 4).

Without the fix can encounter errors as follows: "ERROR: Used length 2097152
greater than bufferLen 2097148 at position 8388592 in table 'xxxx'".

Also removed some related, but unused macro variables, just for cleaning up
codes related to AO storage.
Co-authored-by: NLirong Jian <jian@hashdata.cn>

commit b9b8831a introduced "relation mapping"
infrastructure, which stores relfilenode as "0" in pg_class for shared and
nailed catalog tables. The actual relfilenode is stored in separate file which
provides mapping from OID to relfilenode. gp_replica_check builds hashtable for
relfilnodes to perform lookup for actual files on-disk. So, while populating
this hashtable, consult relmap file to get relfilenode for tables with
relfilenode = 0.

This should help avoid seeing WARNINGs like "relfilenode XXXX not present in
primary's pg_class" or "found extra unknown file on mirror:" for
gp_replica_check.
Co-authored-by: NDavid Kimura <dkimura@pivotal.io>
Co-authored-by: NAshwin Agrawal <aagrawal@pivotal.io>

Changes in 8.4 for statistics computation is comprehensive.
Will fix any fallouts of this change as we observe them.
Current tests show that things are kosher so removing the fix me.

* add RETURNING clause syntax and information to DELETE reference

* add RETURNING clause syntax for INSERT; update output column description for SELECT

* more basic edits to match postgres docs; adding/updating info for AS clause

Co-authored-by: NGoutam Tadi <gtadi@pivotal.io>
Co-authored-by: NJemish Patel <jpatel@pivotal.io>

Commit b3f300b9 introduced the novel idea tracking oldest xmin
among all distributed snapshots on QEs.  However, the idea is not
applicable to QD because all distributed transactions can be found in
ProcArray on QD.  Local oldest xmin is therefore the oldest xmin among
all distributed snapshots on QD.  This patch fixes the maintenance of
oldest xmin on QD by avoiding DistributedLog_AdvanceOldestXmin() and all
the heavy-lifting that it performs.  Calling this on QD was also hitting
the "local snapshot's xmin is older than recorded distributed
oldestxmin" error occasionally in CI.

The prefixes were commonly out of date, misleading as to whom to talk to
should a job fail, and otherwise not an approach developers agree with
anymore for how to bin different tests. The approach being attempted now
is to bin by component.

Furthermore, acronyms are bad so they are avoided more so with the
changes in this commit.
Authored-by: NKris Macoskey <kmacoskey@pivotal.io>

The convention from release pipelines is for packaging jobs to be
labeled as `gpdb_[RC_|]packaging_[platform]`.
Authored-by: NKris Macoskey <kmacoskey@pivotal.io>

It serves no purpose anymore and doesn't aid in visability of the flow
of the pipeline.
Authored-by: NKris Macoskey <kmacoskey@pivotal.io>

Aconyms are confusing and there is no specific reason they need exist in
this context. This does mean someone will need to type the
"ResourceGroups" or "Interconnect" if specifying those groups while
using the gen_pipeline.py script.
Authored-by: NKris Macoskey <kmacoskey@pivotal.io>

The staggered starting was necessary when the performance of concourse
was improved by limiting the spike in started jobs at any one time.
Performance is not as much a driving factor with concourse changes and
with reduction in size of the pipeline. Therefore the staggered starting
idea can be retired.
Authored-by: NKris Macoskey <kmacoskey@pivotal.io>