Because the pg_dump handle the implicit sequence (serial column type)
and explicit sequence differently in different gpdb version, so we
need to detect the sequences which not included in the table dump sql,
then we should create them firstly. And also for all dependent sequence,
setval() to source value after data transferred.
Signed-off-by: NMing Li <mli@pivotal.io>

The persistent field of LOCK struct may cause confusion. Its meaning is the lock can only
be released after the transaction ends.  This commit renames this field as holdTillEndXact
and changes some related function's name. Some comments are also added.

Prior to this commit, gpaddmirrors was missing two bits of work which
were previously done by gpinitsystem/gpcreateseg. When adding mirrors to
a cluster, the pg_hba.conf on primary segments has to be modified to
allow replication connections, e.g. for pgbasebackup. And after the
mirrors are built, the catalog has to be updated to reflect the new
cluster config.
Co-authored-by: NNadeem Ghani <nghani@pivotal.io>
Co-authored-by: NJim Doty <jdoty@pivotal.io>

Co-authored-by: NJason Vigil <jvigil@pivotal.io>
Co-authored-by: NKris Macoskey <kmacoskey@pivotal.io>

Co-authored-by: NJason Vigil <jvigil@pivotal.io>
Co-authored-by: NTrevor Yacovone <tyacovone@pivotal.io>

Code had these two variables (GUCs), serving same purpose. GpIdentity.segindex
is set to content-id, based on command line argument at start-up and inherited
by all processes from postmaster. Whereas Gp_segment, was session level guc only
set for backends, by dispatching the same from QD. So. essentially Gp_segment
was not available and had incorrect value in auxiliary processes.

Hence replaced all usages with GpIdentity.segindex. As a side effect log files
now have correct value reported for segment number (content-id) in each and
every line of file, irrespective of which process generated the log message.

Discussion:
https://groups.google.com/a/greenplum.org/forum/#!msg/gpdb-dev/Yr8-LZIiNfA/ob4KLgmkAQAJ

This patch helps cut-down 8 secs out of 30 secs (for single node gpinitsystem as
part of gpdemo), mostly more for multinode setups.

- Removes explicit 2 sec and 1 sleep.
- Removes explicit call to CHECKPOINT, not required.
- Avoid starting primaries after initdb. As they were started and then shutdown
  pretty soon to start back cluster with required cmdline arguments.
- Add -w to mirror start.

Still more room for improvement and speedup, this just gets us started.

pycrypto version is now identical with the version in
python-dependencies.txt
Authored-by: NTodd Sedano <tsedano@pivotal.io>

Parse version is now identical with the version in
python-dependencies.txt
Authored-by: NTodd Sedano <tsedano@pivotal.io>

We should pass binary_swap_gpdb_centos6 instead of gpdb_src_binary_swap.

We used to implement the memory auditor feature differently on master
and 5X, on master the attribute is stored in pg_resgroup while on 5X
it's stored in pg_resgroupcapability.  This increases the maintenance
effort significantly.  So we refactor this feature on master to minimize
the difference between these two branches.

- Revert "resgroup: fix an access to uninitialized address."
  This reverts commit 56c20709.
- Revert "ic: Mark binary_swap_gpdb as optional input for resgroup jobs."
  This reverts commit 9b3d0cfc.
- Revert "resgroup: fix a boot failure when cgroup is not mounted."
  This reverts commit 4c8f28b0.
- Revert "resgroup: backward compatibility for memory auditor"
  This reverts commit f2f86174.
- Revert "Show memory statistics for cgroup audited resource group."
  This reverts commit d5fb628f.
- Revert "Fix resource group test failure."
  This reverts commit 78b885ec.
- Revert "Support cgroup memory auditor for resource group."
  This reverts commit 6b3d0f66.
- Apply "resgroup: backward compatibility for memory auditor"
  This cherry picks commit 23cd8b1e
- Apply "ic: Mark binary_swap_gpdb as optional input for resgroup jobs."
  This Cherry picks commit c86652e6
- Apply "resgroup: fix an access to uninitialized address."
  This Cherry picks commit b257b344

This guc was removed by accident in
5cc0ec50.

In case we remove the gucs by accident in the future, we added a
testcase to check for the existence of the resgroup gucs.

Same transaction truncate performs unsafe truncation, hence adding test to cover
for the same.

Upstream commit cab9a065 introduced an optimization to truncate tables
in scenarios that permit "unsafe" operations where we don't have to
churn on the relfilenode for the underlying tables. AO table got a free
ride but for the wrong reason.

This patch teaches heap_truncate_one_rel() to perform the unsafe /
optimal truncation on AO tables. This allows us to converge the callers
back to how they look like in Postgres 9.0.

Specifically, we're now able to inline TruncateRelfiles() back into
ExecuteTruncate() .

One caveat introduced by this patch though, is the "optimal" / unsafe
truncation of an AO table can potentially leak some disk space: we are
not performing a real file-level truncate, merely seeking back to offset
0 -- because the aoseg auxiliary table is truncated -- on the next
write, therefore the space after the EOF mark is wasted in some sense.
Co-authored-by: NJesse Zhang <sbjesse@gmail.com>

Missed one reference

[ci skip]
Authored-by: NTodd Sedano <tsedano@pivotal.io>

[ci skip]
Authored-by: NTodd Sedano <tsedano@pivotal.io>

There were scenarios where adding a new partition to a partition table would
cause a negative or duplicate partition rule order (parruleord) value to show
up in the pg_partition_rule catalog table.

1. Negative parruleord values could show up during parruleord gap closing when
   the new partition is inserted above a parruleord gap.
2. Negative parruleord values could show up when the max number of partitions
   for that level has been reached (32767), and there is an attempt to add a
   new partition that would have been the highest ranked partition in that
   partition's partition range.
3. Duplicate parruleord values could show up when the max number of partitions
   for that level has been reached (32767), and there is an attempt to add a
   new partition that would have been inserted between the partition table's
   sequence of parruleord values.
Co-authored-by: NDavid Kimura <dkimura@pivotal.io>

To avoid disk space spamming while running tests and also save some time, patch
intends to avoid core file generation for intentional PANICs caused by tests.

Using `setrlimit()` to achieve the same is output of discussion with Jacob
Champion and Asim Praveen. Alternative option considered was calling
`quickdie()` instead of PANIC, but that suffers from not providing feedback
PANIC string used by test to validate the reason for PANIC.

GNU readline defines the return value of write_history() as "zero if OK,
else an errno code".  libedit's version of that function used to have a
different definition (to wit, "-1 if error, else the number of lines
written to the file").  We tried to work around that by checking whether
errno had become nonzero, but this method has never been kosher according
to the published API of either library.  It's reportedly completely broken
in recent Ubuntu releases: psql bleats about "No such file or directory"
when saving ~/.psql_history, even though the write worked fine.

However, libedit has been following the readline definition since somewhere
around 2006, so it seems all right to finally break compatibility with
ancient libedit releases and trust that the return value is what readline
specifies.  (I'm not sure when the various Linux distributions incorporated
this fix, but I did find that OS X has been shipping fixed versions since
10.5/Leopard.)

If anyone is still using such an ancient libedit, they will find that psql
complains it can't write ~/.psql_history at exit, even when the file was
written correctly.  This is no worse than the behavior we're fixing for
current releases.

Back-patch to all supported branches.

(cherry picked from commit df9ebf1e)
Fixes #4437.

The test can fail if PANIC happens before insert has even reached the intended
point of finishing commit on master. In this case select rightfully returns zero
rows compared to expected 1 row.

This version of ORCA adds missing Xforms that generate Indexed Nested
Loop Join

* docs - clarify settings for low memory queries

* memory_spill_ration range 0-2 for low mem queries

* docs - add a pl/container upgrade procedure

* replace gppkg -r, -i with -u

Use resource owner to do the cleanup of dispatcher and interconnect(#4761)

Commit b3f300b9 eliminated consulting distributed log in
XidInMVCCSnapshot(). This was based on in the QD, the distributed transactions
become visible at the same time as the corresponding local ones, so we can rely
on the local XIDs only. So, given QD never consults distributed transaction log,
lets completely eliminate its creation and maintenance on QD.

Note, during initdb identity (QD or QE) is currently not known and hence still
32k initial zero-filled distributed log file gets created on QD.

"False is not true" is not particularly helpful when looking at the CI.
Dump the diff file as part of the assertion message.

Diff the actual against the expected, instead of the other way around,
and use three lines of unified context.

* docs - add pl/container resource mgmt info

* incorporate review comments

* edits requested by david

because the behivor of pg_dump is different between gpdb 4 and gpdb 5
so the check_sequence function only can work on gpdb 5

ExecBRInsertTriggers() uses the per tuple memory context, which might be
reset and pfree() SEGV.

Signed-off-by: NAdam Lee <ali@pivotal.io>

if the gptransfer with --full option, it should ensure the spcified
database is not existed in the destination cluster. it isn't guaranteed,
so we decide to remove this case
Signed-off-by: NAdam Lee <ali@pivotal.io>

* docs - low memory query config for RQs and RGs

* add missing topic ending tag

* docs - add content for resgroup global shared memory

* some edits requested by david

* memory_spill_ratio max back to 100

* incorporate edits requested from simon