Root partition and internal parent partitions do not contain any data. But since
currently these tables still have valid relfrozenxid, their age keeps
growing. The problem it poses is to bring the age down need to run vacuum on
root, which trickles down and vacuum's each and every child partition as
well. Instead if only leaf if getting modified those can be vacuumed in
isolation and bring the age down and avoid overhead of vacuuming the full
hierarchy.

So, similar to AO, CO, external tables, etc.. for root and parent partitions
record relfrozenxid as 0 (InvalidTransaction) during table creation. This works
since these tables will never store any xids. This would skip them from age
calculation hence eliminate forced need for vacuum on them. Ideally same can be
achieved by defining root and internal parent partition as AO tables but its lot
more work and ddls need modifications.

Alter table change storage type feature is hidden under this guc.  The syntax
ALTER TABLE <tablename> SET WITH (appendonly=true/false) is not documented as
well, since seems this feature is not full ready for primetime. Hence remove the
guc and keep it disabled, anytime in future can code the feature fully and
expose the same.

Github issue #5300 tracks to full enable the feature.

This feature is not ready for primetime yet, so no point testing the same by
enabling the guc. Hence for now removing the tests whenever in future we expose
this feature add and enable tests for the same.

Github issue #5300 tracks to full enable the feature.

Each DTM retry needs a clean error state because the previous error is
already written to csv log and is handled by subsequent retry.

The test hits the PG_CATCH() block in DTM retry logic.  It uncovers a
bug in that part of the code, leading to PANIC due to
ERRORDATA_STACK_SIZE exceeded.

The upper limit on dtx_phase2_retry_count is increased to 15.  That
allows to keep the test simpler by avoiding PANIC due to max retry count
reached.

There already is a PG_TRY() ... PG_CATCH() block to handle such errors
in the DTM retry logic.  This change makes it easier to test the error
handling in the retry logic.

The patch also fixes a bug in dispatcher that invoked CopyErrorData
interface incorrectly, without switching to a memory context other that
ErrorContext.

Getting our own abbreviation right seems like a good thing.

pg_config.h is already included via c.h from postgres.h, individual
files in the backend should not include it separately (especially
not for improved editor support as some comments alluded to).

Commit 3b3adb2b most likely broke
compilation of the SNMP related code on macOS, as it required a
feature test macro flag to enable the right bits in <sys/types.h>.
There however seems to be little to no reason for the feature test
macros to be defined here either, so remove and try to unbreak the
build rather than resurrect.

Also enable snmp support in Travis CI compilation so we get a heads
up the next time it gets broken.

Adds a facility to dump XLOG_HINT records with xlogdump. We have backported
XLOG_HINT xlog records from upstream and this record type never existed at the
time that xlogdump was originally created.

GPDB5 changed relation indexes on disk, and so they are all invalidated
when upgrading from 4. Rather than expecting the user to know what to do
after that mass-invalidation, write a script to perform a REINDEX
DATABASE for every db_name we have, and point the user to it.
Co-authored-by: NAsim Praveen <apraveen@pivotal.io>

- Previously, DNS was queried within the `Ping` utility constructor, so a DNS failure would always raised an exception.
- Now the DNS query is in the standard `run()` method, so a failure from DNS will raise optionally, depending on the `validateAfter` parameter.
- `Command` declared as a new style class so that `super(Ping, self).run()` can be called.
Co-authored-by: NLarry Hamel <lhamel@pivotal.io>
Co-authored-by: NJemish Patel <jpatel@pivotal.io>

* docs - update gpbackup API - add segment instance and update backup directory information.

Also update API version to 0.3.0.

This will be ported to 5X_STABLE

* docs - gpbackup API - review updates and fixes for scope information

Also, cleanup edits.

* docs - gpbackup API - more review updates and fixes to scope information.

The MyProc inDropTransaction flag was used to make sure concurrent AO
vacuum would not conflict with each other during drop phase. Two
concurrent AO vacuum on same relation was possible back in 4.3 and 5X
where the different AO vacuum phases (prepare, compaction, drop,
cleanup) would interleave with each other, and having two AO vacuum
drop phases concurrently on the same AO relation was
dangerous. However, we no longer do that interleaving anymore on
6.0devel. We now hold the ShareUpdateExclusiveLock through the entire
AO vacuum which renders the inDropTransaction flag useless.

The GUC is already removed from source code, no tests should use it.

The patch removes gp_create_index_concurrently GUC.  There is no use setting
this GUC, concurrent index building is not supported in Greenplum.  Greenplum
needs additional work for this feature to behave properly.  Also undone is an
incorrect (and dangerous) attempt to make this feature work in Greenplum.  The
change was dangerous because IndexStmt was dispatched to QEs outside of a
distributed transaction/2PC, which is a recipe for inconsistency.

GitHub issue #5293 is created to track the work needed for this feature.

The compilation failed on OSX/clag with the error "implicit declaration of
function 'MemoryContextInit' is invalid".

According to 72d4a1fa, this utility was slated for removal in v6.
Removing it.
Authored-by: NChris Hajas <chajas@pivotal.io>

According to its help: "Generates a mapping of Servers, Filesystems,
GPDB Filespaces and Storage Pools", which won't work with the new
filespaces. Additionally, it's not used anywhere and not mentioned in
any documentation.
Authored-by: NChris Hajas <chajas@pivotal.io>

Removes gpseginitsb.sh, gpseginitsb.sh, and gpsegstate.sh. These aren't
used by anything.
Authored-by: NChris Hajas <chajas@pivotal.io>

This was used in the old gpcrondump code if the directIO guc was
enabled. Since this code no longer exists in the master branch, this can
also be deleted.
Authored-by: NChris Hajas <chajas@pivotal.io>

GPDB 4.3 uses a slightly smaller TOAST_MAX_CHUNK_SIZE, and we claim to
be able to upgrade those toast tables, but there's currently no support
in tuptoaster:

	ERROR:  unexpected chunk size 8138 (expected 8140) in chunk 0 of 4
	for toast value 16486 in pg_toast_16479 (tuptoaster.c:1840)

Add some tests to flush this out. This adds the
gp_test_toast_max_chunk_size_override GUC, which allows a superuser to
manually reduce the chunk size used by toast_save_datum().

We can handle changes to TOAST_MAX_CHUNK_SIZE by taking a look at the
size of the first chunk. For the full toast_fetch_datum(), we can
optimistically assume that the max chunk size is TOAST_MAX_CHUNK_SIZE,
and then adjust if that turns out to be false. For the random access in
toast_fetch_datum_slice(), though, I haven't tried a similar
optimization -- if we don't know the chunk size to begin with, it seems
like there are too many corner cases to keep track of when jumping into
the middle of a toast relation. So toast_fetch_datum_slice() will always
open the first chunk, which may somewhat impact performance for some
queries.

This patch attempts to get the sequence code as close to upstream as
possible. There is still mix of code from various upstream versions due to past
cherry-picks for bug fixes so its hard to state which exact upstream version it
matches but atleast all the gpdb specific modifications are minimized and
unnecessary code movement has been removed.

Handling sequences in MPP stage is challenging. This patch refactors the same
mainly to eliminate the shortcomings/pitfalls in previous implementation. Lets
first glance at issues with older implementation.

  - required relfilenode == oid for all sequence relations
  - "denial of service" due to dedicated but single process running on QD to
    serve sequence values for all the tables in all the databases to all the QEs
  - sequence tables direct opened as a result instead of flowing throw relcache
  - divergence from upstream implementation

Many solutions were considered refer mailing discussion for the same before
settling on one in here. Newer implementation still leverages centralized place
QD to generated sequence values. It now leverages the existing QD backend
process connecting to QEs for the query to serve the nextval request. As a
result need for relfilenode == oid gets eliminated as based on oid, QD process
can now lookup relfilenode from catalog and also leverage relcache. No more
direct opens by single process across databases.

For communication between QD and QE for sequence nextval request, async notify
message is used (Notify messages are not used in GPDB for anything else
currently). QD process while waiting for results from QEs is sitting idle and
hence on seeing the nextval request, calls `nextval_internal()` and responds
back with the value.

Since the need for separate sequence server process went away, all the code for
the same is removed.

Discussion:
https://groups.google.com/a/greenplum.org/d/msg/gpdb-dev/hni7lS9xH4c/o_M3ddAeBgAJCo-authored-by: NAsim R P <apraveen@pivotal.io>
Co-authored-by: NAshwin Agrawal <aagrawal@pivotal.io>

Currently, notify messages are not used in GPDB by QE's except for sequence
nextval messages. While cleaning the connection for reuse best to remove all the
notify messages as well to be safe than sorry even later if we start using it
for more things.

Authored-by: NDavid Sharp <dsharp@pivotal.io>

-- removes match for "localhost" because a Greenplum cluster defined
with "localhost" as the name of a node will not work
Authored-by: NLarry Hamel <lhamel@pivotal.io>

-- previously, `gpstop -u` used `ssh` for all commands. This changes to
add a conditional for when the host == localhost, and runs the command
without `ssh`
Co-authored-by: NLarry Hamel <lhamel@pivotal.io>
Co-authored-by: NNadeem Ghani <nghani@pivotal.io>

- Remove async session before CREATE FUNCTION
- Change comment format from -- to /* */

* Add resource group bypass memory limit.

- Bypassed query allocates all the memory in group / global shared memory,
  and is also enforced by the group's memory limit;
- Bypassed query also has a memory limit of 10 chunks per process;

* Add test cases for the resgroup bypass memory limit.

* Provide ORCA answer file.

* Adjust memory limit on QD.