PR pipeline is at: https://gporca.ci.pivotalci.info/pipelines/gpdb_pr

- Add new pullrequest_resource gpdb_pr through docker image at `pivotaldata/pr-resource:latest`
- Add github pull request pipeline `pr_pipeline.yml`
Signed-off-by: NKarthikeyan Jambu Rajaraman <krajaraman@pivotal.io>

Shared libraries must be compiled with the fPIC flag
(position-independent code).

Signed-off-by: NTom Meyer <tmeyer@pivotal.io>
Signed-off-by: NAndrew Edstrom <aedstrom@pivotal.io>
Signed-off-by: NTom Meyer <tmeyer@pivotal.io>

Thanks to @edespino for paring up on this seemingly daunting task.

CDB_MOCKING is an old leftover that isn't set anymore, remove the
comment and add whitespace to make the code like upstream in order
to further minimize the risk of merge conflicts.

use asynchronous way to create gang and dispatch command by default.

Clean up some whitespace issues, correct spelling in comments, move
bracing to be uniform with codebase style (as well the lions part of
this file) and remove dead commented out code.

The file pointer is being freed by url_fclose() so we must copy the
members of the struct we need for error reporting before invoking,
the current coding causes a NULL pointer dereference.

Reported by Coverity scanning.

The ereport() macro will now expand to a do { .. } while(0); type
construction which is incompatible with (void) casting of fprintf()
done in yy_fatal_error() by default. Add a level of indirection to
cope with the new construction.

The elog() calls will now be expanded into a do {..} while(0); type
construction, fix up calls which were incompatible with this.

This prevents the postmaster from unexpectedly croaking if postgresql.conf
contains something like:

include 'invalid_directory_name'

Noah Misch. Reviewed by Tom Lane and myself.

Conflicts with GPDB code:
	src/backend/utils/misc/guc-file.l

In commit 71450d7f, we added code to inform
suitably-intelligent compilers that ereport() doesn't return if the elevel
is ERROR or higher.  This patch extends that to elog(), and also fixes a
double-evaluation hazard that the previous commit created in ereport(),
as well as reducing the emitted code size.

The elog() improvement requires the compiler to support __VA_ARGS__, which
should be available in just about anything nowadays since it's required by
C99.  But our minimum language baseline is still C89, so add a configure
test for that.

The previous commit assumed that ereport's elevel could be evaluated twice,
which isn't terribly safe --- there are already counterexamples in xlog.c.
On compilers that have __builtin_constant_p, we can use that to protect the
second test, since there's no possible optimization gain if the compiler
doesn't know the value of elevel.  Otherwise, use a local variable inside
the macros to prevent double evaluation.  The local-variable solution is
inferior because (a) it leads to useless code being emitted when elevel
isn't constant, and (b) it increases the optimization level needed for the
compiler to recognize that subsequent code is unreachable.  But it seems
better than not teaching non-gcc compilers about unreachability at all.

Lastly, if the compiler has __builtin_unreachable(), we can use that
instead of abort(), resulting in a noticeable code savings since no
function call is actually emitted.  However, it seems wise to do this only
in non-assert builds.  In an assert build, continue to use abort(), so that
the behavior will be predictable and debuggable if the "impossible"
happens.

These changes involve making the ereport and elog macros emit do-while
statement blocks not just expressions, which forces small changes in
a few call sites.

Andres Freund, Tom Lane, Heikki Linnakangas

Conflicts with GPDB codebase:
	config/c-compiler.m4
	configure
	configure.in
	contrib/cube/cubescan.l
	contrib/seg/segscan.l
	src/include/pg_config.h.in
	src/include/pg_config.h.win32

When elevel >= ERROR, we add an abort() call to the ereport() macro to
give the compiler a hint that the ereport() expansion will not return,
but the abort() isn't actually reached because the longjmp happens in
errfinish().

Because the effect of ereport() varies with the elevel, we cannot use
standard compiler attributes such as noreturn for this.

Before dispatching a command, we assume the connection is newly created or is reused. For newly created connection, it must be idle, for reused connection, it should have been cleaned up, meanwhile within the internal dispatching, PQsendGpQuery_shared() also do the busy checking and bad connection checking, so pre-checking looks like unnecessary

To test corner cases, we use faultinjector utility to simulate
segment recovery, segment FATAL&ERROR level errors when gangs are creating.

QD need to cancel QEs when
1) QD get a error
2) one or more QEs got error and cancelOnError was set to true.

We want to cancel QEs as soon as possible once above conditions are reached, but considering
the cost of cancelling QEs is high, we want to process as many pending finish QEs as possible
before actually cancel. The original interval before cancelling is 2 seconds which is too
long that users will see an obvious delay before errors are reported, this commit lower
this interval to 100 ms to speed up the cancelling process.

There are a few bugs in InitializeSupportedFunction.
In particular:
* int4le returns int32 but it should return bool.
* date_le_timestamp returns int32 but it should return bool
* in int4_sum, PGGenericFuncGenerator should take arguments < int64_t, int64_t, int32_t >
* in int8pl, PGGenericFuncGenerator should take arguments < int64_t, int64_t, int64_t >

[#130130821]

Currently, when the stats are not existing for a relation, planner
relies on default values instead of running pg_relation_size to save
time. However, this causes planner to produce bad plans. Earlier, the
code had the feature to collect stats for planner if the table did not
had any stats, but it was removed.

This fix re-introduces the feature with the GUC to control whether to use
default values or run pg_relation_size to get actual stats if stats are
not present. By default, the GUC is turned off.

[#129570829]

Enable NOT IN subquery when merge join on that particular data type is not supported [#120384087] (#1099)

Perl versions prior to 5.10 didn't have the \R sequence which handles
all forms of EOL. Since CentOS5 have an old version of Perl unless it
has been separately upgraded the use of this makes atmsort.pm fail and
thus installcheck-good. Replace with a more backwards compatible class.

This change should be part of commit ba13e666,
but was missed check in previously.

motion_listener back

unnecessary calls of it, to avoid redundant data transmit between QD and QE.

cdbconn_disconnect which have not been hit before by ICG test suits

Through the file content we know it uses personal absolute path, which
can't be used by others. DisableXform/EnableXform are already built-in
functions if GPDB is built with Orca. So we can safely delete this file.

This was causing gpinitstandby failure.  Thank you @zeromax007 and @volkovandr
for the report.  Fixes #1079.

Currently PGFuncBaseGenerator and PGGenericFuncGenerator classes have
only one template that takes two argument only, i.e., two datums. These
templates can support only (built-in) functions that take two arguments
as input only (e.g., int4pl). However, there are built-in functions that
take either more or less than two arguments.
This will benefit count and average aggregate function.

* Enhance foreign key check detection for gpcheckcat

We will now do a bidirectional check for the following
catalogs with foreign key check with pg_class:
    pg_attribute, pg_index, and pg_appendonly

This is accomplished by doing a full join based on the foreign keys and
primary keys between pg_class and a catalog mentioned above. Once
detected, we will now output missing catalog infront of our stderr.

* Add unit test for foreign_key check in gpcheckcat

* update the behave test to account for gpcheckcat creating a repair for foreign_key.

* Add new module for foreign key check in gpcheckcat
* get unittest in a working state
* refactor unit test to be "smaller"

Update behave and unit test for gpcheckcat foreign_key_check:

* We are keeping gp_distribution_policy to be checked as part of the
  left join instead of skipping it all together just because it's a
  master only catalog table.
* Add behave test that deletes from segments instead of master for
  foreign key check
* Fix unit test to be able to mock out the full and left join queries
  for forign key check

Add unit test for gpcheck -C option

* Remove pg_constraint check

Due to pg_class limitations where some flags (relhaspkey) are maintained
lazily (updates to some pg_class columns are not enforced).
There is no good way to create a one to one mapping between pg_class
and pg_constraint. So, removing/commenting it out for now

Authors: Nikhil Kak, Chris Hajas, Chumki Roy, and Marbin Tan