to definite the reusing external table by themselves instead of
searching in gpload, which may have bad performance if there are
too many external tables.

Author: David Sharp <dsharp@pivotal.io>
Author: Amil Khanzada <akhanzada@pivotal.io>

In scenarios where pg_statistic contains wrong statistic entry for an
attribute, or when the statistics on a particular attribute are broken,
for e.g the type of elements stored in stavalues<1/2/3> is different
than the actual attribute type or when there are holes in the attribute
numbers due to adding/dropping columns; following two APIs fail because
they relied on the attribute type sent by the caller:

- get_attstatsslot() : Extracts the contents (numbers/frequency array and
values array) of the requested statistic slot (MCV, HISTOGRAM etc). If the
attribute is pass-by-reference or if the attribute is of toastable type
(varlena types)then it returns a copy allocated with palloc()
- free_attstatsslot() : Frees any palloc'd data by get_attstatsslot()

This problem was fixed in upstream 8.3
(8c21b4e9) for get_attstatsslot(),
wherein the actual element type of the array will be used for
deconstructing it rather that using caller passed OID.
free_attstatsslot() still depends on the type oid sent by caller.

However the issue still exists for free_attstatsslot() where it crashes while
freeing the array. The crash happened because the caller sent type OID was of
type TEXT meaning this a varlena type and hence free_attstatsslot() attempted
to free the datum; however due to the broken slot the datums extracted from
values array were of fixed length type such as int. We considered the int value
as memory address and crashed while freeing it.

This commit brings in a following fix from upstream 10 which redesigns
get_attstatsslot()/free_attstatsslot() such than they robust to scenarios like
these.

commit 9aab83fc
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Sat May 13 15:14:39 2017 -0400

    Redesign get_attstatsslot()/free_attstatsslot() for more safety and speed.

    The mess cleaned up in commit da075960 is clear evidence that it's a
    bug hazard to expect the caller of get_attstatsslot()/free_attstatsslot()
    to provide the correct type OID for the array elements in the slot.
    Moreover, we weren't even getting any performance benefit from that,
    since get_attstatsslot() was extracting the real type OID from the array
    anyway.  So we ought to get rid of that requirement; indeed, it would
    make more sense for get_attstatsslot() to pass back the type OID it found,
    in case the caller isn't sure what to expect, which is likely in binary-
    compatible-operator cases.

    Another problem with the current implementation is that if the stats array
    element type is pass-by-reference, we incur a palloc/memcpy/pfree cycle
    for each element.  That seemed acceptable when the code was written because
    we were targeting O(10) array sizes --- but these days, stats arrays are
    almost always bigger than that, sometimes much bigger.  We can save a
    significant number of cycles by doing one palloc/memcpy/pfree of the whole
    array.  Indeed, in the now-probably-common case where the array is toasted,
    that happens anyway so this method is basically free.  (Note: although the
    catcache code will inline any out-of-line toasted values, it doesn't
    decompress them.  At the other end of the size range, it doesn't expand
    short-header datums either.  In either case, DatumGetArrayTypeP would have
    to make a copy.  We do end up using an extra array copy step if the element
    type is pass-by-value and the array length is neither small enough for a
    short header nor large enough to have suffered compression.  But that
    seems like a very acceptable price for winning in pass-by-ref cases.)

    Hence, redesign to take these insights into account.  While at it,
    convert to an API in which we fill a struct rather than passing a bunch
    of pointers to individual output arguments.  That will make it less
    painful if we ever want further expansion of what get_attstatsslot can
    pass back.

    It's certainly arguable that this is new development and not something to
    push post-feature-freeze.  However, I view it as primarily bug-proofing
    and therefore something that's better to have sooner not later.  Since
    we aren't quite at beta phase yet, let's put it in.

    Discussion: https://postgr.es/m/16364.1494520862@sss.pgh.pa.us

Most of the changes are same as the upstream commit with following additional
changes:
- Relcache translator changes in ORCA.
- Added a test that simulates the crash due to broken stats
- get_attstatsslot() contains an extra check for empty slot array which existed
in master but is not there in upstream.
Signed-off-by: NAbhijit Subramanya <asubramanya@pivotal.io>

I used the command `behave test/behave/mgmt_utils --format=steps.usage --dry-run` to identify unused steps. Some of these were false positives or were extra given/when/then steps that I did not remove.

Author: Chris Hajas chajas@pivotal.io

This was found in the dispatch ICG regress test where
ProcessStartupPacket fault injector was being used. In this recent
version of 6.0, FTS probe handler now uses ProcessStartupPacket. In
addition to blocking the backend, this now blocks the FTS probe
handler which caused failover to occur.

Author: Xin Zhang <xzhang@pivotal.io>
Author: Jimmy Yih <jyih@pivotal.io>

This ftsEnabled variable is no longer necessary because FTS is always
enabled now. Back in 4.3.x and 5.x, this variable was a representation
of whether or not the cluster had filerep mirrors.

Author: Xin Zhang <xzhang@pivotal.io>
Author: Jimmy Yih <jyih@pivotal.io>

This is to avoid the cluster failure due to lack of shared memory
reserved for many connections.

Author: Asim R P <apraveen@pivotal.io>
Author: Xin Zhang <xzhang@pivotal.io>

Adding support to create cluster for tinc testing. This also create
gptest database. Here is an example script to run tinc tests:

```
cd ~/workspace/gpdb
source .../greenplum_path.sh
make create-tinc-test-cluster
source gpAux/gpdemo/gpdemo-env.sh
make -C src/test/tinc walrep_2 # to run walrep_2 tinc tests
```

Author: Xin Zhang <xzhang@pivotal.io>
Author: Ashwin Agrawal <aagrawal@pivotal.io>

The existing memory accounting test for OOM was not able to exceed the
OOM bound due to the plan generated by ORCA v2.53.11. Prior to that the
it would hit the OOM limit.  This commit updates the
test query to exceed OOM with the latest version of ORCA and also bump
the version to 2.53.11.
Signed-off-by: NAbhijit Subramanya <asubramanya@pivotal.io>

-updated XFS mount options
-updated GPDB/ipv4_local_port_range information
-added example for setting blockdev.

PR for 5X_STABLE
Will be ported to MAIN

These had been renamed in GPDB a long time ago for some reason. Rename
them back, to reduce merge conflicts.

Makes for nicer merge conflicts.

To reduce diff vs upstream.

This was accidentally duplicated in some earlier merge or backport. It's
in an "#ifdef WIN32" block, so it's dead for GPDB, but let's be tidy.

It was used by file replication, but now it's gone.

Commit 5cc0ec50 changed gpperfmon_log_alert_level into a proper
enum GUC, but failed to update the gpperfmon behave test.

Convert elog() calls to ereport() for user facing errors in setting
GUC values.

As of the merge with PostgreSQL 8.4, we now have builtin support for
enum GUCs. Convert the bespoke GPDB code for managing enum GUCs into
using the enum GUC functionality. There should be no user visible
changes from this except that the error messages are changing.

gp_hashagg_respill_bias has been dead for close to 10 years while
gp_hashjoin_bloomfilter was a leftover from 1fd4d7ee.
gp_hashagg_rewrite_limit was deprecated close to 10 years ago and
made into a no-op.

gp_fault_action and the consumers of gp_backup_directIO* GUCs were
removed with the recent filespaces removal.

Commit ce3153fa, about to be merged from PostgreSQL 9.0 soon, removes
the -w option from pg_regress's "diff" invocation. That commit will fix
all the PostgreSQL regression tests to pass without it, but we need to
also fix all the GPDB tests. That's what this commit does.

Optimistically start the comparison, without waiting the standby to replay
the checkpoint that is issued at the beginning of gp_replica_check. If
there are any recent on-disk data changes, we will find that a data file
doesn't match. But that's OK, we will retry. But in the common case that
there are no recent changes, this greatly speeds up gp_replica_check.
In particular, if you run gp_replica_check on multiple databases, like
"make installcheck" does, the gp_replica_check() call on the first database
will perform a checkpoint, but the calls on subsequent databases will
likely go through without waiting for any more checkpoints.

When build remap info for fields of type described by given
tupledesc, set tupledesc in remapper only when remapping
is required.

This patch removes codegen wholesale from Greenplum.

In addition to reverting the commits involving codegen, we also removed
miscellaneous references to the feature and GUC.

The following commits from 5.0.0 were reverted (topologically ordered):

f38c9064 Support avg aggregate function in codegen
87dcae4c Capture and print error messages from llvm::verifyFunction
65137540 libgpcodegen assert() overrides use GPDB Assert()
81d378b4 GenerateExecVariableList calls regular slot_getattr when the code generation of the latter fails
05a28211 Update Google Test source path used by codegen
22a35fcc Call ereport when code generating float operators
79517271 Support overflow checks for doubles.
b5373a1e Fix codegen unittest link problem
7a1a98c9 Print filename and lineno in codegened CreateElog
58eda293 Fix wrong virtual tuple check in codegened slot_getattr
bc6faa08 Set llvm_isNull_ptr to false when the result of codegened expression evaluation is not null
8bbbd63f Enhance codegened advance_aggregates with support for null attributes
e1fd6072 Abort code generation of expression evaluation trees with unsupported ExprState types
509460ee Support null attributes in codegen expression evaluation framework
739d978d Move enrollment of codegened ExecVariableList to ExecInitNode
c05528d1 Fix CHECK_SYMBOL_DEFINED macro in CMakeLists.txt
12cfd7bd Support offset calculation when there is null in the tuple and codegen is enabled
40a2631e Use slot_getattr wrapper function for regular version.(#1194)
613e9fbb Revert "Fix codegen issue by moving slot_getattr to heaptuple.c similar to"
ee03f799 Fix cpplint error on advance aggregate
2a65b0aa Fix slot function nullptr issue in expr eval.
3107fc0e Fix for !(expr1 & expr2) and hasnull logic in slot_getatrr codegen.
c940c7f6 Fix codegen issue by moving slot_getattr to heaptuple.c similar to Postgres.
c8125736 Introduce guc to enable/disable all generator.
a4f39507 Ensure that codegen unittests pass with GCC 6.2.0 (#1177)
682d0b28 Allow overriding assert() functionality in libgpcodegen in DEBUG mode
3209258a Organize codegen source files and unit tests
d2ba88a9 Fix codegen unittests using DatumCastGenerator
020b71a5 Generate code for COUNT aggregate function
0eeec886 Fixing codegen related bugs in InitializeSupportedFunction
41055352 Rewrite PGGenericFuncGenerator to support variable number of types
e5318b6b Add AdvanceAggregatesCodegenEnroll mock function
87521715 Codegen advance_aggregates for SUM transition function
2043d95b Use string messages in codegened slot_getattr fallback block
f160fa5a Add new GUC to control Codegen Optimization level.
697ffc1a Fix cpplint errors
c5e3aed4 Support Virtual Tuples and Memtuples in SlotGetAttrCodegen
5996aaa7 Keep a reference to the CodegenManager in code generators
6833b3c2 Remove unused header and just include what you use in codegen
ab1eda87 Allow setting codegen guc to ON only if code generation is supported by the build
dcd40712 Use PGFuncGeneratorInfo to codegen pg functions
83869d1c Replace dynamic_cast with dyn_cast for llvm objects
23007017 Decide what to generate for ExecEvalExpr based on PlanState
387d8ce8 Add EXPLAIN CODEGEN to print generated IR to the client
c4a9bd27 Introduce Datum to cpp cast, cpp type to Datum cast  and normal cast.(#944)
66158dfd Record external function names for useful debugging
adab9120 Support variable length attributes in SlotGetAttrCodegen.
335e09aa Proclaim that the codegen'ed version of generated functions are called in debug build
50fd9140 Fix cpplint errors
88a04412 Use ExprTreeGeneratorInfo for expression tree generation.
3b4af3bb Split code generation of ExecVariableList from slot_getattr
8cd9ed9f Support <= operator for date/timestamp data types and some minor refactors.
e4dccf46 Implement InlineFunction to force inline at call site
71170942 Mock postmaster.o for codegen_framework_unittest.t
09f00581 Codegen expressions that contain plus, minus and mul operators on float8 data types
d7fb2f6d Fix codegen unittests on Linux and various compiler warnings while building codegen.
45f2aa96 Fix test and update coding style This closes #874
1b26fbfc Enrolled targetlist for scan and aggregate
ebd1d014 Enhance codegen framework to support arbitrary expr tree
ec626ce6 Generate and enroll naive ExecEvalExpr in Scan's quals
1186001e Revert "Create naive code-generated version of ExecQual"
6f928a65 Replace RegisterExternalFunction with GetOrRegisterExternalFunction using an unordered_map in codegen_utils
6ae0085b Move ElogWrapper to GpCodegenUtils.
d3f80b45 Add verifyfunction for generated llvm function.
7bcf094a Fix codegen compiler error with 8.3 merge
aae0ad3d Create naive code-generated version of ExecQual
dce266ad Minor code quality fixes in src/backend/codegen
d281f340 Support null attributes in code generated ExecVariableList
82fd418e Address a number of cpplint errors in codegen_utils_unittest.cc
887aa48d Add check for CodegenUtils::GetType for bool arrays
bb9b92c6 Enhance Base Codegen to do clean up when generation fails
b9ef5e3f Fix build error for Annotated types
a5cfefd9 Add support for array types in codegen_utils.
2b883384 Fix static_assert call
7b75d9ea This commit  generates code for code path: ExecVariableList > slot_getattr > _slot_getsomeattrs > slot_deform_tuple. This code path is executed during scan in simple select queries that do not have a where clause (e.g., select bar from foo;).
6d0a06e8 Fix CodeGen typos and CodeGeneratorManagerCreate function signature in gpcodegen_mock.c
4916a606 Add support for registering vararg external functions with codegen utils.
ae4a7754 Integrate codegen framework and make simple external call to slot deform tuple. This closes #649
ee5fb851 Renaming code_generator to codegen_utils and CodeGenerator to CodegenUtils. This closes #648
88e9baba Adding GPDB code generation utils
Signed-off-by: NJesse Zhang <sbjesse@gmail.com>
Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>
Signed-off-by: NSambitesh Dash <sdash@pivotal.io>

-gpexpand - Add link to AdminGuide, Expanding a Greenplum System
-pljava_vmoptions - change set classification to reload from restart

These were added in GPDB to encapsulate RelFileNode and persistent table
information. But persistent tables are no more, so there is no reason to
differ from upstream.

To reduce merge conflicts as we start to merge with upstream again.

Also add some comments, based on the descriptions of the original bugs
that were fixed.

These are no longer used for anything.