This reverts commit 8c62e892.

Make get_stack_depth_rlimit() handle RLIM_INFINITY more sanely.

Rather than considering this result as meaning "unknown", report LONG_MAX.
This won't change what superusers can set max_stack_depth to, but it will
cause InitializeGUCOptions() to set the built-in default to 2MB not 100kB.
The latter seems like a fairly unreasonable interpretation of "infinity".
Per my investigation of odd buildfarm results as well as an old complaint
from Heikki.

Since this should persuade all the buildfarm animals to use a reasonable
stack depth setting during "make check", revert previous patch that dumbed
down a recursive regression test to only 5 levels.

In commit 47e11a2d we unintentionally
included changes to submodules, revert this part.

Signed-off-by: NGang Xiong <gxiong@pivotal.io>

CID 169225: Dereference after null check (FORWARD_NULL)
var_deref_model: Passing null pointer onerel to AppendRelToVacuumRels, which
dereferences it.
In commit b3abb1b7 if
`open_relation_and_check_permission()` fails during dropPhase, we try to open
the relation again by passing the dropPhase flag set to false so that it can
perform the cleanup phase. We must check if onerel is NULL again after the
second call to `open_relation_and_check_permission()` before proceeding
further.

Since Travis doesn't build the Orca libraries, disable Orca in the
build since the default is now to enable Orca. This enables us to
keep running Travis as an alternative CI, albeit with non-default
options.

This should work with both dbid and contentid backup filenames.
Signed-off-by: NKaren Huddleston <khuddleston@pivotal.io>

Based on the experiments with tpch and tpcds, we found that 16MB would
be the optimal size of mdcache.
[#96490982].

Vacuum of AO tables happens in four phases: prepare, compact, drop and cleanup.
When there are concurrent transactions executing vacuum on AO tables, if one of
the vacuum transaction is executing a drop phase, the other transaction will
not be able to open the relation if it is also executing the drop phase. When
the relation open fails, vacuum simply exits without performing the cleanup
phase. The relfrozenxid in pg_class is updated during the cleanup phase and
hence the age of the table does not get reduced. This becomes a problem because
the age of the database does not reduce and eventually the xid wraparound issue
will be encountered.

This problem is fixed by performing the cleanup phase even if the open relation
fails during drop phase.

The column_to_scan function used to get the smallest eof column was
operating incorrectly. It would loop the segfiles until it found a
segfile with nonzero tuplecount, grab the first column that has a
nonzero eof, and return that value. The issue here is that it did not
iterate through all the columns to find the smallest nonzero eof. With
this commit, we now loop through all the columns of the segfile to
decide on a column to return. The unit tests for this function were
also incorrect so we fix that in this commit too.

Due to the different phases of VACUUM, compaction phase could have
committed but the drop was cancelled. This could leave some
aoseg/aocsseg entries in AOSEG_STATE_AWAITING_DROP state which could
then be cleaned up in a subsequent VACUUM as expected... but it
currently just skips. The master keeps a hash table (AppendOnlyHash)
and uses it during VACUUM to check tuple count, state, and
aborted/committed. Our issue here is that state checking was only
being done against state AWAITING_DROP_READY. It was not including
COMPACTED_AWAITING_DROP in the state check. This commit adds the check
and updates the detailed comment block on the state transitions in
appendonlywriter.h.

When doing an ADD COLUMN operation on an AOCO table, the vpinfo byte
array stored in the aocsseg auxiliary table for each column is updated
to add an extra vpinfo index to match the relnatts value in
pg_class. However, entries in the aocsseg auxiliary table with state 2
(AOSEG_STATE_AWAITING_DROP) will not have their vpinfo byte array
updated. Subsequent calls to getAOCSVPEntry function will then fail
due to the vpinfo size does not match the relnatts value in pg_class
(e.g. persistent table rebuild). Since we have the AccessExclusiveLock
on the table during ADD COLUMN, we might as well schedule the drop
before the vpinfos are updated to prevent any inconsistency issues.

We expect lazy VACUUM to skip drop phase if it cannot acquire the
relation's AccessExclusiveLock. This commit properly sets the dontWait
flag to True when calling try_relation_open during drop phase. We also
change the dontWait part of try_relation_open to not error out because
callers should be the ones to handle an invalid relation.

If GPDB is configured to build with orca, this commit adds checks to
ensure orca libraries are available.

We also chose more representative headers for the checks.

If gpdb is configured to build with orca it requires a Greenplum patched
version of Xerces library. This commit adds an explicit check for the
patched Xerces.

This also gives us an opportunity to be more idiomatic in doing
autoconf: we are replacing the manual manipulation of LIBS in
src/backend/Makefile with `AC_CHECK_LIB` in `configure.in`.

When the GPDB is build with GPORCA, `optimizer` GUC is set to on
by default. If GPDB is not build with GPORCA this guc is off by default.
Signed-off-by: NOmer Arap <oarap@pivotal.io>

This changes autoconf to enable orca by default. To be clear, the twin
flags of `--enable-orca` and `--disable-orca` are already available,
this commit merely changes the default (and help messages accordingly).
Signed-off-by: NOmer Arap <oarap@pivotal.io>

As of commit c7ff7663, subplans are initialized even they are unused. In some
case, the generated subplan is not used, but it has motionId = 0, which breaks
executor's sanity check. This patch fixes the issue by removing unused
subplans.

See discussions at:
https://github.com/greenplum-db/gpdb/issues/2383

To test if `ON SEGMENT` is compatible with all the other options.

Previously, we remove the shared memory object when drop resource group,
and restore it if the transaction aborts. The concurrently access to
shared memory object would fail in this way.
If the resource group is dropped, new transactions to this resource
group will be queued up until the drop transaction is finished.
Signed-off-by: NRichard Guo <riguo@pivotal.io>

Partition pruning test in ICW is failing because of `max_stack_depth` is
too small for sles11 environment. This commit increases the limit from
`100kB` to `500kB`.

Until now, people working on documentation would indicate that their
commits did not need to be run through automated testing by using the
`[ci skip]` annotation on their commit messages. This automates the same.

Signed-off-by: NDivya Bhargov <dbhargov@pivotal.io>

This change was made in order to facilitate the new git branching rules.
In order to use different credentials to pull in from our fork, we need
to be able to more finely control which git repo we are pulling from
when we clone the pipeline to smoke test a change.
Signed-off-by: NDivya Bhargov <dbhargov@pivotal.io>

Signed-off-by: NDivya Bhargov <dbhargov@pivotal.io>

The old implementation of function remove_unused_initplans is buggy, we can
come up with multiple ways to break it, because it doesn't utilize postgres
plan/expression tree walker correctly. Rewrite the whole function to resolve
found and potential bugs.

CID 129288, 129277:
Buffer not null terminated (BUFFER_SIZE_WARNING)
buffer_size_warning: Calling strncpy with a maximum size argument of 1024 bytes
on destination array m->path2 of size 1024 bytes might leave the destination
string unterminated.
This issue was detected in the `xlog_mm_redo()` function. Fixed by using
`strlcpy()` to ensure that the buffer is null terminated.

CID 129856: Argument cannot be negative (NEGATIVE_RETURNS)
negative_returns: fd is passed to a parameter that cannot be negative.
This issue was also found in the `xlog_mm_redo()` function.
Fixed the issue by making sure that `close()` is not called on negative fd.

CID 129947: Out-of-bounds write (OVERRUN)
overrun-local: Overrunning array buf of 80 bytes at byte offset 80 using index
nbytes (which evaluates to 80).
The above defect was found in `WalRcvUsr2Handler()` which is in gpdb only and
not in upstream.

CID 129271, 129270, 129269, 129268 : Function address comparison (BAD_COMPARE)
func_conv: This implicit conversion to a function pointer is suspicious:
The above defect was found in `WalReceiverMain()`. The comparison with NULL was
being performed against a function name and not a function pointer or variable
which is clearly incorrect. Since we do static linking of the functions, the
check was not necessary in the first place. Fixed the defect by removing the
check.

The problem is that if ExecPartitionSelector() gets no tuples from the child
node, it doesn't call InsertPidIntoDynamicTableScanInfo() at all. It should
call InsertPidIntoDynamicTableScanInfo() in that case, with InvalidOid as the
partition OID, to signal to PartSelectedExpr that it was executed and no
partitions are selected.

Now with this change it will call InsertPidIntoDynamicTableScanInfo() after the
last last tuple has been read, always, not only when there were no tuples at
all.

* gpdeletesystem and pg_dumpall ref page updates

* clarify that --no-gp-syntax removes table distribution clauses

Remove some unnecessary null-tests, and replace a goto-label construct
with an "if" block.

Michael Paquier, reviewed by me.

Coverity points out that mdc_finish returns a pointer to a local buffer
(which of course is gone as soon as the function returns), leaving open
a risk of misbehaviors possibly as bad as a stack overwrite.

In reality, the only possible call site is in process_data_packets()
which does not examine the returned pointer at all.  So there's no
live bug, but nonetheless the code is confusing and risky.  Refactor
to avoid the issue by letting process_data_packets() call mdc_finish()
directly instead of going through the pullf_read() API.

Although this is only cosmetic, it seems good to back-patch so that
the logic in pgp-decrypt.c stays in sync across all branches.

Marko Kreen

We were not storing sourceTag in CachedPlanSource, so if a cached plan
is used later, that field is broken, and would break resource queue
concurrency limit functionality for example.

Reported by issue #2308 when using JDBC/ODBC

* docs - add plan cache invalidation info

* use correct postgres version, remove reuse statement