libgpos has a set of banned API calls which needs to be allowed with
the ALLOW_xxx macro in order for gpopt to compile (and thus run).
The changes to ereport() brought a need for allowing abort() since
it now invokes abort when building with --enable-cassert.

This is a temporary fix awaiting the removal of the banning of
function calls entirely. Pushed even though the CI pipeline failed
to provide a clean run (for seeminly unrelated reasons) due to the
absence of this blocking other efforts.

Signed-off-by: NGang Xiong <gxiong@pivotal.io>

bugbuster test spi
Signed-off-by: NGang Xiong <gxiong@pivotal.io>

When PRs are submitted, trigger a build and then run ICG tests. 
Its output will be placed in separate folder from other pipelines.
Signed-off-by: NGang Xiong <gxiong@pivotal.io>

* We were using string comparison for our version checks and did not account for two digit numbers.
  For example:
  comparing '5.0.10.0' to '5.0.5.0' will return 5.0.5.0 being greater

  Switching to magic number comparison, fixing this issue.

* Add unit test for gppkg

Commit fixes issues for incremental recoverseg
reported by @zeromax007 and @volkovandr.
- as part of commit 7d639c30,
function FtsFindSuperuser was removed. But it deep inside was
setting the user. Missing to set the same causes the error
for queries to compact change tracking logs.
- Also, as part of commit a453004e,
function gp_changetracking_log was broken, resulting in SIGSEGV
on incremental gprecoverseg. Fixed to avoid passing NULL to
heap_compute_data_size.

Fixes #1161 and Fixes #1207.

Previously, gprecoverseg reported an error if mirror states were
mismatched during a full recovery.  Now, a full recovery will always
correct the mirror state, so that the check is no longer necessary.

#132695977

The second command should not be executed after the first one failed.
Or else, the make command (e.g. installcheck) will return 0 even
when there's failure.
Signed-off-by: NGang Xiong <gxiong@pivotal.io>

Relcache init file was failing to load because of duplicate occurrence of
OpclassOidIndexId in the nailed indexes list.  Consequently, relcache was
always built from scratch upon every child process startup.  Also added
OperatorOidIndex from upstream to the nailed index list, to match upstream
(8.3), as pointed out by Heikki.
Signed-off-by: NAsim R P <apraveen@pivotal.io>

This is due to the 8.3 merge removed the implicit casting to text.
Detailed info can be found in following commit:

    commit a453004e
    Merge: 5dc7dda9 d13f41d2
    Author: Heikki Linnakangas <hlinnakangas@pivotal.io>
    Date:   Wed Jun 15 15:17:19 2016 +0300

    One big user-visible change included is the removal of implicit cast from
    other datatypes to text. When this was done in PostgreSQL, it caused a lot
    of sloppily written application queries to break. Which is a good thing in
    the long run, but it caused a lot of pain on upgrade.
Signed-off-by: NKuien Liu <kliu@pivotal.io>

Related commit:

    commit 9e1e324d
    Author: Heikki Linnakangas <hlinnakangas@pivotal.io>
    Date:   Tue Aug 2 00:59:25 2016 +0300

        Remove obsolete relaosegrelid and relaosegidxid fields from pg_class.

        They were not used for anything. Look at segrelid and segidxid in
        pg_appendonly instead.

[#132551801]
Signed-off-by: NXin Zhang <xzhang@pivotal.io>

* Add new task in perf pipeline to setup GPDB

* Install GPDB from an installer copied over to the instance
* Add remote_push and remote_pull for easy copying
* Fix failure when remote_run_script is called when EXPORTS is empty

Authors:
- Shreedhar Hardikar <shardikar@pivotal.io>
- Omer Arap <oarap@pivotal.io>

This patch refactors ReadBuffer_common to remove relErrMsgString
argument and use relpath of reln->smgr_rnode instead.

Author: Jimmy Yih

PQconnectPoll() must *not* be called, until the socket is writeable, otherwise PQconnectPoll() assumes that
the connect() has finished and sends startup packet to it which is actually not connected yet.

For big chunk, chunk info is freed before the chunk is actually reallocated. If gp_realloc() get failed, existing code do not rollback related info and pfree() will report double free issue due to inconsistent info

Explicitly enable/disable codegen/orca while running installcheck-good and installcheck-bugbuster (#1213)

While running installcheck , we are not explicitly setting codegen or orca to "off" . This causes confusion and also is prone to bugs where the feature may be turned "on" before running installcheck.

This commit explicitly turns off the feature during installcheck-good and installcheck-bugbuster.

Thanks,
Nikhil & Nikos

This is a followup test for PR #1005

The log_newpage() routine allocates an XLog record on the stack with
uninitialized persistentTid and persistentSerialNum fields.  Insertion of these
XLog records causes problems in XLog replay during recovery.

This commit ensures that the XLog records written by log_newpage() always have
the persistentTid and persistentSerial set correctly.

This commit also makes the test
src/test/tinc/tincrepo/mpp/gpdb/tests/storage/crashrecovery/test_reindex_pg_class.py
pass.

Signed off by: Asim Praveen <apraveen@pivotal.io>, Amil Khanzada
<akhanzada@pivotal.io>

* Enable to build with Netbackup client libraries 7.6
* Remove all libgpbsa shared objects when doing clean
* Add netbackup behave 7.6

There is no functional changes in this commit.

Authors: Chris Hajas and Chumki Roy

gprecoverseg shouldn't recover segments with persistent mirroring disabled

If a cluster with mirrors was rebuilt with the gp persistent rebuild tool without
mirrors, catalog inconsistencies may be present. gprecoverseg should
not attempt to recover these segments, but should still recover segments
without this inconsistency.

Authors: Chris Hajas, Chumki Roy, Karen Huddleston

In commit 8df78e67, there were some
CaQL removal. The scan that replaced the CaQL call was using the wrong
dbid.

Author: Jimmy Yih
Reported by @volkovandr in https://github.com/greenplum-db/gpdb/issues/1202

Summary:
* Properly propagate the C++ compiler from autoconf
* Disable aggressive loop optimizations in GCC 4.8+

This commit is in the spirit of postgres/postgres@105f3ef4 , but done
for C++ code so that the GPORCA translator can properly build when the
compiler is GCC 4.8+

Fixes greenplum-db/gpdb#1186 .

TL;DR, here is the additional context on what specific to GPORCA translator:

GCC 6.2.0 compiler (which is required for codegen) got aggressive loop optimization to remove loop when enumerating variable length array.

In code `CTranslatorRelcacheToDXL.cpp`, we have a loop to enumerate all
the attributes of an index:

```
INT iAttno = pgIndex->indkey.values[ul];
```

Where `values[ul]` is of following data structure:

```
typedef struct
{
...
	int2		values[1];		/* VARIABLE LENGTH ARRAY */
} int2vector;					/* VARIABLE LENGTH STRUCT */
```

This is standard way of dealing with variable length array in Postgres. However, optimizer
thinking there is only 1 element, hence removed the loop.

This caused wrong result when using GPDB with GPORCA. The repro is
```
-- multi-column index
create table index_test (a int, b int, c int, d int, e int,
  constraint index_test_pkey PRIMARY KEY (a, b, c, d));

insert into index_test select i,i%2,i%3,i%4,i%5 from generate_series(1,100) i;

-- expecting using index scan with index cond: c=5
explain select * from index_test where c = 5;
```

When compile with GCC 6.2.0, the `index scan` is not selected, but a `table scan` is chosen
instead. The root cause is the compiler removed the loop, and only keep the first column.

The fix is to add $CXXFLAG auto detection to add the
`-fno-aggressive-loop-optimizations` for $CXX compilers if that version of compiler supporting
such optimization. Hence to avoid having the above issue for gpopt component.

Signed-off-by: NNikhil Kak <nkak@pivotal.io>

Signed-off-by: NNikos Armenatzoglou <nikos.armenatzoglou@gmail.com>

GPDB currently has 2 APIs of creating work files
  1. Using workfile_mgr :

    1. This uses a workfile set to contain all the related workfiles.
    2. All workfiles are created under the same directory. As a result,
       when it is time to delete the workfiles, we can simply delete the directory.
    3. There is no mechanism to control the exact name of the workfile created.
       All workfiles are referenced using an ID. `"%s/spillfile\_f%u", work\_set-\>path, file\_no);`
    4. Deletion of workiles is automatically handled in case of an exception,
       as all the files are contained in one directory, thus no counter/files are leaked.

  2. Using ExecWorkfile :

    1. There is a Create/Open/Read/Write/Flush/Close API.
    2. This is the lower-level API also used by workfile_mgr to manage the workfiles directly.
    3. Each operator is resposible to correctly close the workfile it creates.

  When we write to a workfile, we reserve and add the number of bytes to used_segspace,
  and add that amount to `workfile->size`. Then, we call `workfile_update_in_progress_size`.
  If the workfile has an owning workset, we update the workset's `in_progress_size` counter.
  However, if it has no such workset, we completely ignore the count -
  and this is what ultimately leaks the `used_segspace` counter.

  This is a fix to account for bytes written in workfiles that are not part of any workfile set -
  that is those created using the lower level API. Hence when we abort or error out,
  we subtract this amount from the `used_segspace` counter.

Authors:
Shreedhar Hardikar <shardikar@pivotal.io>
Xin Zhang <xzhang@pivotal.io>
Karthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>
Nikos Armenatzoglou <nikos.armenatzoglou@gmail.com>

* Escape distribution key when creating WET in gptransfer

* There is an issue where when a user creates a table and the table has
  a column with CAPITAL LETTERS or quotes and that column is used as the
  distributed key, then gptransfer fails. gptransfer tries to create
  the WRITABLE EXTERNAL TABLE for gpfdist with unquoted values and
  breaks; this is because the DISTRIBUTED BY portion is not quoted properly.
* This commit handles the quotation for the distributed key

Authors: Marbin Tan & Todd Sedano

Slot_getattr in header file cause to have different pointer.
This cause issue when we register as external function. LLVM
expect the name of the external function to be unique for
each pointer. In order to fix it, we introduced wrapper function
in codegen_wrapper.cc and use that instead of slot_getattr.

This reverts commit c940c7f6.

When using a docker image, we noticed that gprecoverseg will fail due to
parsing stdout that has a SSH warning line such as "Warning: Permanently
added [host] to the list ...". We now ignore these lines until the
desired output begins.

Authors: Larry Hamel and Chris Hajas